Use profit as the main signal, then check revenue and conversion to understand why one threshold performed better. Treat a displayed leader as early and directional until every evidence check passes.
What the metrics mean
Question: Did more shoppers order?
Use it to understand whether the threshold made shoppers more or less likely to complete checkout.
Question: Did order value improve?
Use it to see whether shoppers bought more, bought less, or changed cart size.
Question: Which side kept more per exposure?
Use profit per exposure as the primary comparison after shipping cost and margin assumptions.
Question: Leader or final winner?
A leader is directional. A final winner appears only after time, volume, confidence, and traffic-quality checks pass.
Question: Is the result stable?
Use it to avoid reacting to a short spike.
Question: Is there enough reliable data?
Track full days, exposures on both sides, confidently matched orders, confidence, and traffic quality.
Common result patterns
- If conversion drops slightly but profit increases, the higher threshold may still be better because shoppers are placing larger orders.
- If conversion improves but profit falls, the lower threshold may be too expensive because the store is covering more shipping cost.
- If revenue and profit are close, wait for more orders before changing your long-term threshold.
Why results can lag
Results appear after shoppers see a test side and completed orders can be confidently connected back to it. A store can have live traffic before there are enough matched orders to make the comparison useful.
If a result looks slow, wait for more traffic, check that the offer is live, and confirm the banner is visible where shoppers can see it.
How to use the report
Check profit first
A higher threshold can lift profit even if conversion moves slightly.
Review revenue and conversion
These show whether shoppers are buying more, buying less, or changing cart size.
Check readiness
Wait for at least 7 full days, 1,000 exposures per side, 200 confidently matched orders in total, and the remaining quality checks.
Confirm deliberately
Review margin, shipping cost, seasonality, and customer expectations before applying any change.
After the decision
A test-threshold winner updates the live threshold only after confirmation. A control winner leaves the live offer unchanged. A store-rates result pauses the tested free-shipping offer after confirmation. Completed comparisons remain in history so you can revisit the evidence and decide when conditions justify another test.