Home & Landing
Highlighting the Money-Back Guarantee A/B Test Example
Making an existing refund guarantee easier to notice may reduce purchase hesitation. Additional orders can also bring more refunds and processing costs. This test changes the guarantee’s visibility, not its terms, and needs enough follow-up time for refunds to occur.
A/B test scenario. This page is a test plan, not the result of a real experiment.
- Home & landing
- Net revenue per visitor
Control vs Variant
How hard the same message is pushed is the variable.
- Changed
- Emphasis
- Page
- Home & landing
- Difference
- changed
Hypothesis
If the money-back guarantee already on the page is clearly highlighted, visitors weighing the risk may find it easier to decide. If the revenue from extra orders holds up after refunds, that could raise net revenue per visitor.
Test plan
When to run this test
Pre-purchase worry about risk is easiest to see in support tickets asking about refund terms; session recordings of visitors hunting for guarantee details and survey answers about the fear of not liking the product reflect the same concern. Once the guarantee is easy to notice, hesitation should drop, purchases should rise and refunds should not cancel out the gain. First make sure you can actually honor the guarantee you plan to highlight. If it only matches the statutory minimum return right, do not present it as a special marketing benefit.
How to measure success
Net revenue per visitor
Compare revenue after returns per assigned visitor over a fixed period; assess transaction costs separately.
Also track
- Purchase conversion rate (CR)Check whether purchase rate changes.
- Return rateReview returns after an adequate follow-up period and assess their effect on profitability.
- Return processing costMake sure the operational workload remains manageable.
- Repeat purchase rateCompare later purchases over the same follow-up period in both arms.
How to set it up
- EmphasisChoose one way to make the existing guarantee more prominent.
- TermsKeep coverage, duration, eligibility and claim process unchanged.
- WordingUse the same accurate terms; a conditional versus unconditional guarantee is a separate test.
- EconomicsRead net revenue alongside return-processing costs.
- Customer historyReview first-time and repeat purchasers separately.
What to watch when reading the result
- 1Even if the purchase conversion rate goes up, decide on net revenue per visitor after refunds, since extra orders can bring more refunds. If the return rate or return processing cost eats up the sales gain, more orders alone do not justify shipping the variant.
- 2When the guarantee is emphasized, its terms must stay readable; do not accept a win from a variant that shrinks or hides the terms. Review first-time and repeat purchasers separately, and track the repeat purchase rate for the same length of follow-up in both arms.
- 3Refunds show up over time, so do not name a revenue winner before the follow-up window ends; an early read shows only the sales side. Once the window closes, check the conversion rate gap with the A/B test significance calculator; it does not test net revenue per visitor, so treat a small revenue gap with the same caution.
Frequently Asked Questions
Why use net revenue per visitor instead of conversion?
Additional orders can also produce more refunds. Use revenue after returns per assigned visitor as the primary outcome and assess refund-handling costs separately.
When should the guarantee result be read?
Wait for comparable return follow-up to mature. An early reading can capture sales but miss refunds. Use equal windows for repeat-purchase outcomes as well.
What limits apply when emphasizing a guarantee?
Only promise what can be honored and keep conditions readable. Do not portray mandatory statutory rights as a special benefit. Review relevant new and repeat-buyer groups.
More tests in Home & Landing
All testsTesting a Language-Adapted Greeting on the Homepage
Home & landingPrimary action completion rate
Adapt only the greeting to the browser language and compare it with the default-language greeting, while keeping the rest of the page unchanged.
Testimonial Carousel or All Quotes Visible A/B Test
Home & landingPrimary action completion rate
A testimonial carousel against every quote visible at once. Examine both arms, what happens to cards past the first and the metric being tracked.
Showing Social Follower Count A/B Test
Home & landingPurchase conversion rate (CR)
Show a current social follower count beside the brand link and compare it with no count, tracking conversion and trust signals.