Showing a Visible No Thanks Option on an Offer A/B Test

Session recordings of users hunting for a way to close an optional offer, offer-related complaints, or exits right after it appears suggest users are unsure how to get past it. If the idea is right, first-display acceptance may drop, but users who see they are in control accept the offer more often on later visits. Keep the option readable and clearly visible, and do not re-show the offer to someone who declined in the same session. If users find the offer irrelevant, check who sees it first; never test making decline harder for legally required consent such as marketing permission or cookies.

Total offer acceptance rate

Count acceptances among all eligible assigned users over a predefined period; do not count repeat acceptances twice.

Also track

• First-impression acceptance rateMeasure acceptance on the first offer display.
• Repeat visit rateCompare all assigned users over the same follow-up period; do not select only those who declined.
• Offer-related complaintsTrack complaints about the visibility of the decline option or repeated offers.
• Withdrawal rateIf withdrawal is possible for this offer, compare later withdrawals over the same period.

• Decline optionAdd a visible “Not now” option to the optional offer, which already has an exit.
• Keep constantKeep the offer, acceptance action, eligibility and existing exit behavior unchanged.
• Follow-upUse the same observation period and repeat-offer policy in both arms.
• AudienceReview new and existing users separately.
• Separate testsTest decline wording and repeat-display timing separately.

1. 1Base the decision on total offer acceptance across an equal follow-up period for both arms, and do not stop the test early because first-display acceptance falls; the hypothesis expects some acceptance to shift later. Even if total acceptance rises, more offer-related complaints or a higher withdrawal rate mean you should not make the variant permanent.
2. 2Compare the repeat visit rate across all assigned users over the same observation period, not only among those who declined. Review new and existing users separately; the result applies to this offer and this repeat-offer policy, so test again before using it on other offers.
3. 3An unclear gap in total acceptance may simply mean acceptances after the first display have not built up yet. Do not read the result before the observation period ends or change that period mid-test. Before launch, use the sample size calculator with your current total offer acceptance rate to find the users needed per arm, then plan the time to reach that number plus the observation period.