Highlighting the Search Term in Results: A/B Test Idea

Plan this test when you notice visitors cannot tell which result fits their query, such as a new query submitted right after results load, session recordings of a cursor moving back and forth between titles, or exits from the results page with no click. When highlighting does its job, visitors pick the matching result sooner and need fewer follow-up queries. If search logs show irrelevant results ranking near the top, ranking may be the real problem; fix search quality first, because highlighting will not make up for it.

Product page visit rate

Compare product-page visits per eligible assigned visitor who sees the same search results; clicks alone do not prove the right product was found.

Also track

• Purchase conversion rate (CR)Compare purchases among all eligible assigned visitors over the same follow-up period.
• New query rateCompare visitors who submit another query after seeing results, using the same results-impression denominator.
• Search-results exit rateCompare exits among visitors assigned on the search-results page.

• HighlightUse one fixed, accessible style to mark query-matching words in result titles.
• DensitySet how multiple matches are shown and check that titles remain readable.
• ScopeHighlight titles only; highlighting descriptions is a separate test.
• RelevanceReview weak matches without changing ranking during the test.
• Device and queryCheck truncated mobile titles and predefined query-length groups.

1. 1An increase in product page visit rate alone does not show that visitors found the right product. If purchase conversion rate and new query rate do not move the same way, or search-results exit rate climbs, the highlight may be pulling weak matches into view.
2. 2Make sure the highlight does not rely on color alone and that contrast and a second visual cue set it apart. Keep matching narrow enough that it does not mark most of a title; if titles become hard to scan, you are measuring that setting rather than highlighting itself.
3. 3This comparison speaks only to the highlight style you tested, and only in titles; highlighting descriptions is a separate comparison. Do not carry a finding from single-word queries over to long ones, and read the predefined query-length groups and truncated mobile titles separately.
4. 4With no clear overall gap, an effect in one query group may be getting lost in another. Before reading groups separately, use each group's product page visit rate as the baseline in the sample size calculator and compare the sessions it requires per variant with what the group actually gets.