# Promo Message on the App Loading Screen A/B Test Idea Page: https://alidemirbas.com.tr/en/lab/ab-test-library/tests/promo-message-on-the-app-loading-screen-ab-test-idea ## Hypothesis If the loading screen shows one short message about the current campaign instead of only the brand mark and indicator, visitors may notice the offer during a wait that happens anyway. If the message does not lengthen loading, the purchase conversion rate may go up. ## When to run this test If the loading screen already stays up for a while, it may be worth using that moment for the campaign, especially when promoted categories get few visits, session recordings skip campaign areas, or shoppers tell support they learned of the campaign late. If the message gets noticed, visitors who see it move on to promoted categories and products more often, and that interest carries through to purchases. Do not launch unless the campaign is live and the message states its terms accurately, and measure first that heavy imagery does not slow loading. ## Primary metric **Purchase conversion rate**: Compare purchases among eligible visitors assigned to each loading-screen message; do not restrict analysis to visitors who waited long enough to see it. ## Other metrics - **Post-load CTA click-through rate**: Track CTA clicks, but do not use them as a substitute for purchases. - **Campaign product purchase rate**: Compare purchases of promoted products among all eligible assigned visitors. - **Page exit rate**: Compare exits during loading in both arms. - **Time to main content readiness**: Measure time until the page is ready for interaction consistently in both arms. ## What to check before and during the test - **Message**: Show one short message about the existing campaign during loading. - **Offer**: Keep campaign terms, targets and other promotional areas unchanged. - **Presentation**: Keep the loading indicator and surrounding design unchanged. - **Discovery**: Review visits to promoted categories and products. - **Visibility**: Interpret results against observed loading time without artificially extending it. ## What to watch when reading the result - Post-load CTA clicks do not stand in for purchases. Judge the variant on purchase conversion rate across all eligible assigned visitors; looking only at visitors who waited long enough to see the message overstates its value. - If time to main content readiness grows or exits during loading rise in the variant, the message is adding to the wait, and a small lift in purchases is no reason to ship. Campaign visuals must not cover the loading indicator either; without it, visitors may think the app has frozen. - The finding is tied to this campaign and this message, so do not expect the same effect once the campaign changes. You can look at predefined contexts such as device or connection, but do not treat groups built afterward from loading time as if they were randomized arms. - A short message may move purchases too little for this traffic to detect when the gap is unclear. Compare the sample per variant that the [sample size calculator](https://alidemirbas.com.tr/en/calculators/sample-size-calculator) gives for the current purchase conversion rate and the lift you care about with the visitors each arm actually received. If the test fell well short, do not read the result as proof of no effect. ## Frequently Asked Questions ### Can loading be prolonged to show the message? No. Fit the message into genuine waiting time. Measure actual duration, and do not treat groups selected by treatment-affected duration as equivalent randomized populations. ### Can analysis include only visitors who waited long enough to see it? No. Use purchasing across all assigned eligible visitors. Selecting those who remained introduces selection bias and can overstate the message’s value. ### What does increased post-loading CTA clicking mean? It is diagnostic, not a purchase outcome. Use purchase conversion as the primary outcome and examine category visits to understand navigation.