Problem vs Solution Headline Framing A/B Test Example

Plan this test when the homepage headline does not seem to match why visitors arrive, such as a high bounce rate from certain traffic sources, survey answers from visitors who did not understand the offer, or search terms that mix descriptions of the problem with searches for the product by name. Expect the framing that fits visitors' awareness to bring more completed actions without lowering the qualified lead rate. If the problem is the offer itself, a headline test will not fix it. Write both headlines to match the ad copy, without inflating the problem into fear and without promising to solve something the product does not address.

Primary action completion rate

Compare completion of the same predefined action under the two headline approaches.

Also track

• Bounce rateMake sure a framing mismatch does not drive visitors away.
• Scroll depthTrack continued browsing; scrolling is not proof that visitors read the content.
• Time on pageRead time alongside completion; a longer visit is not automatically better.
• Qualified lead rateMake sure the share of leads that meet the target profile does not fall.

• FramingCompare problem-focused and solution-focused headlines for the same offer.
• Keep constantKeep subheading, hero image, CTA and supporting copy unchanged.
• Traffic contextReview existing source or audience segments instead of inferring awareness from the outcome.
• DisplayKeep both headlines readable across devices.
• Separate testChanging problem-solution framing in both headline and subheading is a separate approach.

1. 1Pick the winning headline by primary action completion rate; longer time on page or deeper scrolling does not show that a headline was better understood. If one headline brings more actions but a lower qualified lead rate, that result is not enough to ship it.
2. 2This offer, these two headlines and the traffic mix in the test bound the finding. Read source and audience segments separately, and test again in that source before carrying a difference seen in one traffic source over to other sources or to traffic arriving from different ad messages.
3. 3If the overall difference is small, the two framings may be offsetting each other across segments. Decide which segments you will read before launch and work out the visitors each one needs with the sample size calculator; treat a segment difference found afterward as a lead, not a conclusion.