Shipping Testing helps you find the right shipping rates for your business by testing different shipping experiences against your current configuration. Instead of guessing how changes to shipping costs will affect your store, you can measure the impact on conversion rate, average order value (AOV), revenue, and profit before making permanent changes.
Common experiments include:
Testing different flat-rate shipping prices (for example, $7.99 vs. $5.99)
Testing different free shipping thresholds (for example, free shipping over $50 vs. over $75)
Comparing different shipping rate tiers based on order value
How Shipping Testing works
When you start a Shipping Test, Shogun temporarily serves shipping rates at checkout for customers included in the experiment.
Customers are randomly assigned to one of two experiences:
Original — your current shipping rates
Variant — the shipping rates you've configured for the test
As customers shop and complete purchases, Shogun measures the performance of each experience so you can compare metrics like conversion rate, revenue, average order value, and profit.
Your existing shipping configuration isn't permanently changed while the test is running.
Requirements
To create a Shipping Test, your store must have:
A Shogun plan that includes Shipping Testing
A Shopify plan that supports third-party calculated shipping
Existing shipping profiles and rates configured in Shopify
If your store doesn't meet these requirements, Shipping Testing may be unavailable until they're addressed.
Creating a Shipping Test
To create a Shipping Test:
From your dashboard, create a new Shipping Test.
Give your test a name.
Select the shipping profiles and zones you want to include.
Configure the Variant shipping rates.
Choose your traffic split.
Review your settings and start the test.
You can:
Change flat-rate prices
Adjust free shipping thresholds
Create shipping tiers based on order value
Add or hide shipping rates within the Variant
Carrier-calculated shipping rates are displayed for reference but can't be edited directly in a Shipping Test.
Validate your test
After starting a test, we recommend confirming that the correct shipping rates appear at checkout.
Each running test includes preview links for both the Original and Variant experiences, so you can verify that customers see the expected shipping rates.
Because Shopify may take a few minutes to propagate shipping updates, allow a short delay after starting a test before validating.
Pausing or ending a test
When a Shipping Test is no longer running (paused or stopped), the test immediately stops serving, and your original shipping rates are restored.
If you pause a test:
The experiment stops collecting data.
Customers immediately return to your original shipping rates.
You can resume the test later.
If you end a test:
The experiment stops permanently.
Customers immediately return to your original shipping rates.
The test can't be resumed.
Applying a winning variant
Shipping Tests don't automatically apply the winning shipping rates to your store.
If you'd like to keep the winning configuration:
End the test.
Open Shopify Admin → Settings → Shipping and delivery.
Update your shipping rates to match the winning Variant.
This gives you full control over when permanent changes are made to your store.
Best practices
For the best results:
Validate your Original and Variant experiences after starting a test.
Allow a few minutes for Shopify to propagate shipping changes before checking the checkout.
Avoid editing shipping settings in Shopify while a Shipping Test is running, as this can interfere with the experiment.
Run tests long enough to collect meaningful traffic before making a decision.
Limitations
Shipping Testing currently supports:
One Variant per test (Original vs. one Variant)
Existing Shopify shipping profiles and zones
Please note:
Carrier-calculated shipping rates can't be directly edited.
Winning Variants aren't automatically applied when a test ends.
Customers who have already reached checkout before a test is paused or ended may complete their purchase using the shipping rates they were originally shown.
Frequently asked questions
Does pausing a Shipping Test restore my normal shipping rates?
Yes. Pausing immediately stops serving the test experience and restores your original shipping rates. If you resume the test later, the experiment begins serving traffic again.
What happens when I end a Shipping Test?
Ending a test also restores your original shipping rates. If you want to keep the winning shipping configuration, you'll need to manually update your shipping settings in Shopify.
Can I test multiple shipping variants?
No. Shipping Tests currently compare your Original shipping rates against a single Variant.
Can I edit carrier-calculated shipping rates?
No. Carrier-calculated shipping rates are managed by the shipping provider and can't be edited directly in a Shipping Test.
Why can't I create a Shipping Test?
Shipping Testing requires:
A supported Shogun plan
A Shopify plan with third-party calculated shipping support
Existing shipping profiles and rates configured in Shopify