How to run mutually exclusive tests

When two experiments can reach the same visitor, allowing a visitor into both at once distorts your results. Mutually exclusive tests prevent this: a visitor assigned to one experiment is kept out of the other. This guide covers two setups.

Testing different stages of a user journey

A visitor typically moves through several stages - a landing page, then a product page, then checkout. You may want a separate test at two of those stages, with both tests receiving traffic but no visitor counted in both. Suppose Test 1 runs on the landing page (/) and Test 2 runs on the product page (/product).

Test 1 (/, the landing page) - set traffic allocation to any value below 100%. Visitors not assigned to Test 1 remain eligible for Test 2.
Test 2 (/product, the product page) - set traffic allocation to 100%, or any value higher than Test 1.
In the Advanced Settings of either test, under Experiment exclusions, exclude the other test. The exclusion applies to both, so a visitor in Test 1 is never assigned to Test 2.

Test 1 takes its share, and everyone else flows through to Test 2, with no overlap.

The traffic allocation control in an experiment's settings — Traffic allocation

Another experiment selected under Experiment exclusions — Experiment exclusion

Running multiple tests on the same page

You can also split traffic between two separate tests on one entry point instead of running a single experiment with more variations. For example:

Test 1 - Control: /page-1, Variation: /page-2
Test 2 - Control: /page-1, Variation: /page-3

Testa's integration script evaluates experiments in a random order and assigns the visitor to the first one they qualify for. To split traffic evenly:

Set both tests to 100% traffic allocation.
In the Advanced Settings of Test 2, under Experiment exclusions, exclude Test 1.

This produces an even split because:

The script processes the first experiment (chosen at random) and assigns the visitor to it.
The visitor is not assigned to the second experiment, because of the exclusion.
Since the order is random, traffic divides roughly 50/50 between the two tests.

To weight the split, lower one test's traffic allocation below 100%.