Ali Demirbaş - Lab
TRGet in touch

Home & Landing

Matching the Headline to the Traffic Source A/B Test

Matching a headline to the promise that brought someone to the page may make the next step clearer.

  • Home & landing
  • Primary action completion rate

Control vs Variant

Who the visitor is decides what fills the slot.

Changed
Personalization
Page
Home & landing
Difference
changed

Hypothesis

A bad match can show irrelevant copy, and managing many sources adds complexity; overly personal wording may feel like tracking. Compare the rule with a fixed headline within the same traffic sources.

How to run this test

Primary metric

Primary action completion rate

Randomize eligible incoming traffic and compare the predefined completed action; do not compare different traffic sources as if they were experiment arms.

Other metrics

  • Bounce rateMake sure a message mismatch does not increase exits.
  • Conversion by sourceReview which sources show a gain and which do not.
  • Qualified lead rateMake sure source-matched copy does not increase the share of unqualified leads.
  • Page load timeMake sure personalization does not add delay.

What to check before and during the test

  • MatchingDefine the source-to-headline match before launch.
  • Keep constantKeep CTA, offer, images and source classification unchanged.
  • FallbackShow a predefined general headline when the source is unknown.
  • LoadingCheck whether the headline visibly changes during loading.
  • ScopeReview results by source before expanding the match rules.

Setup mistakes to avoid

  1. 1Do not infer source from personal data or imply that you know more about a visitor than they expect.
  2. 2Define a fallback headline so unknown sources do not leave the page blank.
  3. 3Do not change headline and CTA copy together by source.
  4. 4If personalization happens client-side, check for visible headline flicker.
  5. 5Do not roll out a win from one source to all sources without reviewing each.