Shogun supports multiple types of experiments.
Since these tests can run simultaneously, Shogun must decide which test a shopper (visitor) participates in when multiple tests apply to the same visit.
This is controlled by the traffic distribution algorithm: Greedy or Even.
Single Test Participation Rule
Each shopper is assigned to only one A/B test at a time.
This ensures:
Clean, reliable results
No overlapping experiences
No conflicting changes on the same page
đ Even if multiple tests are live, a shopper will only participate in one.
đ§ Algorithm 1: Greedy
Greedy means a shopper is assigned the moment they first become eligible for any test. If multiple tests are eligible at that moment, one is chosen at random.
How it works
When a shopper visits the store, Shogun checks which tests are currently active on the page. If only one test is running â for example, a Theme Test â the shopper is automatically assigned to that test.
If multiple tests are active, such as a Theme Test and a Home Page Test, Shogun follows a âfirst seenâ rule. The first test the shopper encounters captures them. Once that assignment is made, the shopper wonât be added to any other tests later, even if they qualify for them.
This approach mimics how users naturally move through a store â whichever experiment they hit first is the one they end up participating in.
Example
Scenario: A Theme Test (running site-wide) and a Home Page Test (specific to the homepage).
When a shopper lands on the homepage:
Both tests are active.
Shogun identifies that the shopper is eligible for both.
The shopper is randomly assigned to one of the two tests at a 50/50 ratio.
After the assignment, they will not be able to take the other test later.
Notes
When multiple tests apply, random assignment ignores each testâs internal variation split.
For example, a Page Test might have a 70/30 split and a Theme Test might have a 50/50 split, but between the two tests, the split remains 50/50.
If a shopper first lands on a page not under test (for example, a blog page) while a Theme Test is running, theyâll be placed into the Theme Test since itâs the first applicable test.
A shopper can only participate in one test at a time. Once assigned, they will not be reassigned elsewhere.
âď¸ Algorithm 2: Even
Even means that shoppers are assigned to tests in equal proportions, regardless of âfirst seenâ logic.
How it works
In Even mode, each shopper is deterministically assigned to a test using their session ID. This ensures that all tests are equally likely to receive traffic, regardless of where the shopper enters the funnel.
Once the shopperâs test is determined, their assigned variant within that test is selected according to the configured traffic split (for example, 70/30). This approach guarantees a consistent, balanced distribution across all active tests, preventing top-of-funnel pages from dominating.
Example
Scenario: A Theme Test (site-wide), a Home Page Test, and a Product Page Test are all running.
When a shopper lands on the homepage:
Shogun checks the shopperâs pre-assignment.
Even though the homepage test appears first, the shopper will not automatically enter it.
If the shopper was pre-allocated to the Product Page Test, they will skip the homepage test.
When they later visit a product page, they will enter their assigned Product Page Test as expected.
Notes
Even distribution ensures all tests receive balanced, predictable traffic across the funnel.
Pre-assignment occurs before the shopper engages with any page.
Once assigned to a test, the shopper will only see variants from that test and will not enter others.
Internal variant splits (for example, a 70/30 split within a test) are still respected after assignment.
đ§ Key Takeaways
Shogun always enforces one active test per shopper.
Greedy prioritizes the first test encountered, mimicking the real-world browsing order.
Even prioritizes balanced traffic distribution across all running tests.
When a Theme Test overlaps with Page/Template Tests, Shogun randomizes the assignment between them to avoid double exposure.
How pre-assignment affects experiment metrics
When we use pre-assignment, weâre simply deciding in advance which variant a session would see if it becomes eligible for the experiment. That eligibility still requires the visitor to actually reach the page or object being tested and meet any audience rules.
If a visitor is pre-assigned to a variant, but:
leaves the site before reaching the tested page, or
navigates somewhere that isnât part of the experiment,
Then they are not counted in the experiment at all. They wonât appear in session counts, conversion metrics, or any reporting for that test.
Why does this not break even distribution:
Even distribution is enforced only among eligible participants, meaning visitors who actually see the tested experience. Any traffic that drops off before reaching the test is simply excluded, not âlostâ to one variant or another. As a result, the visitors who do qualify for the test are still split evenly and measured correctly.
Cache Management
Shogun stores a visitorâs test variant assignment in local storage, ensuring they see the same test version each time they return.
Key details:
For theme and template tests, dynamic scripts ensure the correct variant is served regardless of Shopifyâs default caching.
Shogunâs metafield and optimizer configuration on Shopify are refreshed approximately every 12 hours to ensure shops are running the latest experiment setup and optimizer code.
Audience Targeting in Concurrent Tests
Each test evaluates audience conditions independently, meaning a visitor might qualify for one test but not another.
Targeting rules include:
Device type (e.g., mobile, desktop)
Visitor status (new vs. returning)
Geographic location
Login status
URL pattern matching
Referrer sources (e.g., campaigns, social)
This modular targeting helps you run isolated or layered tests without unintentionally mixing segments.