Ali Demirbaş - Lab
TRGet in touch

Mobile App

A/B Test Idea: Home Screen for Returning App Users

Returning visitors may want to resume a recent task without navigating from the beginning.

  • Mobile app
  • Revenue per visitor (RPV)

Control vs Variant

Who the visitor is decides what fills the slot.

Changed
Personalization
Page
Mobile app
Difference
changed

Hypothesis

A continue block may help, while incorrect classification or stale activity data can undermine it. The defined variant is a block for returning users, not a redesign of the new-user homepage.

How to run this test

Primary metric

Revenue per visitor (RPV)

Compare revenue per assigned visitor within predefined new and returning segments, then report the overall result.

Other metrics

  • Homepage click-through rateMeasure interaction with the first block; do not classify an impression as unnoticed just because it was not clicked.
  • Return-to-cart rateTrack eligible returning visitors who resume their carts.
  • New-user conversionCheck whether misclassification or the fallback experience harms new-user conversion.
  • Page load timeMake sure personalization does not harm LCP.

What to check before and during the test

  • Continue blockAdd the predefined continue block for returning visitors in the variant.
  • New visitorsKeep the new-visitor homepage unchanged in both arms.
  • ClassificationUse one fixed signal and classify visitors before assignment.
  • Content scopeKeep other blocks and recommendation logic unchanged.
  • FallbackDefine the default experience for unknown or misclassified visitors.

Setup mistakes to avoid

  1. 1Do not change segment definitions during the test.
  2. 2Do not expose personal information such as name or address on the homepage.
  3. 3Keep the default experience usable for visitors in the wrong segment.
  4. 4Do not change segment rules and content blocks together.
  5. 5Do not bypass caching in a way that slows the page.