Ali Demirbaş - Lab
TRGet in touch

Cart & Checkout

Raising the Free Shipping Threshold A/B Test

Changing the free-shipping threshold may lead some shoppers to add items and others to abandon their carts.

  • Cart
  • Revenue per visitor (RPV)

Control vs Variant

Where the line sits is the variable.

Changed
Threshold
Page
Cart
Difference
changed

Hypothesis

Compare revenue per visitor and consider order completion and shipping costs; the result applies to the specific threshold tested.

How to run this test

Primary metric

Revenue per visitor (RPV)

Compare net revenue per assigned visitor across the two shipping thresholds, accounting for order conversion and shipping cost.

Other metrics

  • Average order value (AOV)Compare revenue per completed order; interpret it alongside conversion and RPV.
  • Order completion rateCompare completed orders among all visitors assigned to each threshold.
  • Cart-to-checkout rateCompare assigned cart visitors who start checkout.
  • Shipping cost per orderTrack shipping expense and its effect on contribution margin.

What to check before and during the test

  • ThresholdCompare the current free-shipping threshold with one alternative chosen from unit economics and shipping costs; set both values before launch.
  • Keep constantKeep prices, promotions, shipping message and its presentation unchanged.
  • RevenueUse RPV to assess whether any increase in AOV offsets a change in conversion.
  • CostInclude shipping expense and margin impact in the decision.
  • AudienceReview new and returning visitors with the same definitions; keep each visitor in the same variant.

Setup mistakes to avoid

  1. 1Hold each threshold constant throughout the experiment.
  2. 2State the free-shipping conditions accurately.
  3. 3Choose the alternative using the current basket distribution and shipping costs.
  4. 4Do not change prices, promotions or how the threshold is presented during the same test.
  5. 5Do not treat a higher AOV as a win if RPV or margin falls.