Run your first test

Choose the comparison

Open Optimize and select your published offer. The control is your current setup; the alternative is the change you want to test. For example, compare a $75 threshold with $90. Change one thing so you can interpret the difference.

A store-rates comparison is available for an offer with one enabled tier. That side uses the store's shipping rates without the tested free-shipping offer. Other store discounts and shipping rules still matter.

Review costs and readiness

Enter the margin and average shipping cost requested by the app. Profit reporting depends on these assumptions. Resolve the readiness messages before starting; do not try to bypass a targeting or campaign conflict.

Keep your theme, other promotions, and targeting stable during the comparison. One active experiment runs at a time.

Start and verify the comparison

Review the control, alternative, target market, and cost assumptions, then use the start action in Optimize. Check that the experiment appears as running and that exposures begin to collect. Shoppers are assigned to a side; repeatedly refreshing your own browser is not a reliable way to preview both variants.

New experiments run for up to 13 weeks, with confidence checked weekly. Measurement ends early if either offer qualifies as a statistically supported winner. You choose which offer to apply. Experiments already running under an older method retain their original schedule.

Let the evidence collect

Before a winner can qualify, a test needs at least seven full days, 1,000 mature exposures per side, and 200 confidently matched orders in total. These minimum checks are not a traffic target. Confidence and traffic-quality checks must also pass. These are minimum gates, not a promise that a winner will appear when you cross them. An exposure is an eligible cart assigned to a side, including carts that do not lead to an order. Recent exposures need time for orders to arrive before they count toward a confidence check.

The first scheduled check is after seven days, then weekly for up to 13 weeks. The app accounts for repeated checks and requires a meaningful profit difference before declaring a winner. A temporary lead can reverse, so avoid stopping just because a favorable number appears.

Dashboard totals and charts update daily during measurement. Confidence updates at the scheduled weekly checks. Check the displayed update time when comparing figures.

See how traffic and effect size change the planning estimate. The 14-day free trial is time to set up and start collecting data; it does not guarantee a conclusive result.