# Matching the Headline to the Traffic Source A/B Test

Page: https://alidemirbas.com.tr/en/lab/ab-test-library/tests/matching-the-headline-to-the-traffic-source-ab-test

## Hypothesis

If the headline picks up the promise of the ad or referring page that brought the visitor in, visitors may see sooner that they landed in the right place and move on with more confidence. Compared with a fixed headline, the matched one could bring a higher primary action completion rate.

## When to run this test

Bring this test forward when you can see a clear gap between ad wording and the headline, especially if specific campaigns show high bounce or session recordings catch visitors turning back from the first screen. If the match works, visitors who see a matched headline bounce less and complete the predefined action more often. Build the match only from source details such as the campaign or referring page; do not infer anything from personal data or imply you know more about visitors than they expect. If the gap comes from the offer itself, align the ad with the page offer first.

## Primary metric

**Primary action completion rate**: Randomize eligible incoming traffic and compare the predefined completed action; do not compare different traffic sources as if they were experiment arms.

## Other metrics

- **Bounce rate**: Make sure a message mismatch does not increase exits.
- **Conversion by source**: Review which sources show a gain and which do not.
- **Qualified lead rate**: Make sure source-matched copy does not increase the share of unqualified leads.
- **Page load time**: Make sure personalization does not add delay.

## What to check before and during the test

- **Matching**: Define the source-to-headline match before launch.
- **Keep constant**: Keep CTA, offer, images and source classification unchanged.
- **Fallback**: Show a predefined general headline when the source is unknown.
- **Loading**: Check whether the headline visibly changes during loading.
- **Scope**: Review results by source before expanding the match rules.

## What to watch when reading the result

- Compare the primary action completion rate among visitors randomized within the same traffic sources; do not compare sources against each other as if they were arms. If actions rise while the qualified lead rate falls or bounces climb, the match may be setting the wrong expectation or showing some visitors an irrelevant headline, so do not roll it out on that gain.
- Read the result source by source: a gain in one source may not repeat in others, so decide on the match for each source separately. Page load time matters too; if the headline arrives late or flickers, the difference may come from the implementation rather than the message.
- With a small or unclear gap, start by looking at how sources are spread across the arms, because an uneven mix can hide a real effect or create one that is not there. Also check that the visitor split matches the planned ratio. Then test the primary action completion rates within each source with the [A/B test significance calculator](https://alidemirbas.com.tr/en/calculators/ab-test), and do not count small per-source gaps as wins unless they are significant.

## Frequently Asked Questions

### Can different traffic sources serve as the experiment arms?

No. Randomize within a common eligible pool, with balanced assignment by source if useful. A source-composition difference must not be mistaken for a headline treatment effect.

### What should appear when the source is unknown?

Show a predefined generic fallback headline. Fix the source-to-message mapping before launch rather than leaving unknown visitors without a coherent heading.

### Why might a personalized headline unsettle visitors?

An overly personal implication or incorrect mapping can feel intrusive or irrelevant. Use completed main actions as the primary outcome and monitor the predefined exit guardrail.

### Can a headline change during page loading affect results?

Yes. Visible flicker and loading delay become part of the experience. Check them directly and do not generalize one source’s result to all acquisition channels.