Pop-Up Timing on Category Pages: A/B Test Idea

Purchase conversion rate

Compare purchases among all eligible visitors assigned before the pop-up trigger, including visitors who never see it.

• Pop-up response rateReport reach and response separately because the two triggers may expose different numbers of visitors.
• Pop-up CTA click-through rateMeasure clicks on the pop-up's primary button.
• Page exit rateCompare exits from pages where the pop-up was eligible to appear.
• Session durationUse session duration only as context; longer sessions are not automatically better.

• TriggerCompare display on the first eligible page view with display after the visitor has scrolled 50% of that page in the same session.
• Keep constantUse the same offer, design, size, position and frequency cap.
• AssignmentAssign visitors before the trigger; keep those who never see the pop-up in the primary analysis.
• Primary outcomeReview purchases and exits alongside pop-up interactions.
• Separate testsTest the offer, design and frequency cap separately.

1. 1Do not show the pop-up immediately on page load.
2. 2Avoid full-screen pop-ups on mobile.
3. 3Keep the close control visible and usable.
4. 4Do not show the same pop-up repeatedly on every page.
5. 5Do not run multiple pop-ups at the same time.