UI Elements
Announcement Bar Timing: Immediate or Delayed A/B Test
Showing the campaign banner on page load presents it immediately. Waiting until visitors scroll 50% of the page gives them time to engage first, but those who do not reach that point will not see it. Compare the two timings while keeping the banner, placement, frequency, and campaign action the same.
A/B test scenario. This page is a test plan, not the result of a real experiment.
- UI elements
- Campaign page visit rate
Option A vs Option B
The moment is the variable, not what appears.
- Changed
- Timing
- Page
- UI elements
- Difference
- changed
Hypothesis
If the banner appears on page load, every visitor who opens the page can get the campaign message right away; if it waits for the scroll threshold, it reaches fewer visitors, but ones already engaging with the page. Which timing leads to a higher campaign page visit rate may depend on whether that engagement makes up for the reach the delayed version loses.
Test plan
When to run this test
A banner shown on load that gets dismissed right away, session recordings of visitors pausing when it appears, or a delayed banner that never reaches a large share of visitors all suggest timing shapes campaign page visits. If delayed timing works, visitors who see the banner move on to the campaign page more readily, enough to offset those who never reach the threshold; if immediate display works, the gap in reach wins out. If most visitors leave before the threshold, the delayed version's reach is narrow from the start, so check scroll depth first.
How to measure success
Campaign page visit rate
Compare campaign-page visits among all eligible assigned visitors, including those who never reach the delayed trigger.
Also track
- Announcement bar reachCompare the share of assigned visitors who see the bar.
- Unintended click rateTrack unintended clicks associated with the bar's appearance.
- Overall purchase conversion rate (CR)Compare purchases among all eligible assigned visitors.
- Page exit rateCompare exits using the same session window.
How to set it up
- TimingCompare display on page load with display after the visitor scrolls 50% of the same page.
- TriggerUse 50% scroll as the only delayed trigger; do not add a timer or inactivity rule.
- PlacementKeep the bar in the same stable position in both versions.
- FrequencyKeep the frequency cap and dismissal behavior unchanged.
- DeviceCheck mobile timing and whether the bar obstructs content.
What to watch when reading the result
- 1Announcement bar reach explains how timing changes who sees the bar, but it is not the decision metric. Use campaign page visit rate to decide, keeping delayed-group visitors who never see the bar in the denominator, and check that overall purchase conversion, page exit rate and unintended click rate do not worsen.
- 2Reserve space so the bar does not push content when it appears, and keep its content the same in both versions. Do not show a dismissed bar again in the same session or trigger the delayed bar while a visitor is in the middle of another action; otherwise the difference reflects the interruption, not the timing. Keep the conclusion to the page you tested, and read mobile timing and overlaps separately.
- 3Keeping visitors who never reach the threshold in the denominator dilutes the effect, so the difference may look small. Check whether the gap in campaign page visit rate is beyond chance with the A/B test significance calculator; a diluted gap that does not reach significance does not on its own show that timing has no effect.
Frequently Asked Questions
Should delayed-group visitors who never see the strip be excluded?
No. They remain in the denominator. Failing to reach the scroll threshold is part of timing's real effect. Compare completion of the predefined campaign goal across all eligible assigned visitors.
Which trigger defines the delayed version?
It appears at 50% page scroll in this test. Another trigger would be a different comparison. Keep content, placement, frequency and target action identical.
What risk does a strip appearing later create?
It can shift content and cause accidental clicks. Reserve space or use an equally stable position. Monitor accidental clicks and review mobile timing and overlaps.
What does the exposure rate tell me?
It explains how timing affects access to the strip. It does not determine success. Check the target action, overall conversion and page exits.
More tests in UI Elements
All testsNotification Badge: Number or Dot A/B Test Example
UI elementsVisit rate for the indicated section
An experiment example comparing a numbered notification badge with a plain dot. See both arms, the urgency versus pressure reasoning and the KPI.
Adding a Live Chat Button to the Page: A/B Test Idea
UI elementsPurchase conversion rate (CR)
A live chat button on screen against no button at all, written up as an experiment example with both arms, the space trade-offs and the tracked KPI.
Auto-Opening the Live Chat Window: A/B Test Example
UI elementsPurchase conversion rate (CR)
A closed chat button in the corner versus a window that opens by itself. This experiment example covers both arms, the interruption risk and the KPI.