# Relevance vs Best-Selling Default Search Sort A/B Test

Page: https://alidemirbas.com.tr/en/lab/ab-test-library/tests/relevance-vs-best-selling-default-search-sort-ab-test

## Hypothesis

With the best query matches on top, visitors may find what they want sooner; with best sellers on top, undecided visitors may settle on a popular option more easily. Which default wins on search purchase conversion rate could depend on how clearly visitors know what they are looking for.

## When to run this test

It is time to compare the two defaults when visitors on search results often switch away from the default sort, or skip the top products and scroll down, and when certain queries pile up searches with no interaction. The better default draws more clicks on the first four results and leaves fewer visitors changing the sort. If the problem lies in the result set itself, with irrelevant products showing up for a query, fix the matching first, because no ranking method will solve that.

## Primary metric

**Search purchase conversion rate (CR)**: Compare purchases within the same follow-up window among eligible visitors who see search results; do not select the primary analysis based on post-assignment search behavior.

## Other metrics

- **Clicks on the first four results**: Measure clicks on the first four product links using the same search event denominator.
- **Sort changes**: Compare visitors who switch from the default sort to another option.
- **Average order value**: Report order value using the same discount and cancellation rules; track gross margin separately.
- **Searches with no interaction**: Count queries where results were shown but no product click or other predefined search action occurred.

## What to check before and during the test

- **Search conversion**: Compare relevance-based default ranking with best-selling default ranking for the same result set.
- **Top results**: Compare clicks on the first four results in each arm.
- **Sort changes**: See which default visitors change more often.
- **Order value**: Review order value and gross margin alongside conversion.
- **Segments**: Review new and returning users separately.

## What to watch when reading the result

- Even if one arm shows higher search purchase conversion, do not decide without checking average order value and gross margin, since best-seller ranking can shift sales toward different products. Clicks on the first four results and searches with no interaction explain the gap but do not pick the winner.
- Run the main analysis on visitors who reach search results and were assigned before they saw either ranking; if you later narrow it to clickers or buyers, the arms stop being comparable. Read new and returning users separately, since each group may lean toward a different default. Before carrying the result over to category page sorting, test it there separately.
- Before reading the result, confirm that sort options stayed visible in both arms, the filter set did not change and out-of-stock products did not rise to the top. If sponsored products were not clearly marked apart from organic results, the gap cannot be credited to the ranking method.
- If the gap is small, enter each arm's visitors who saw search results and those who purchased into the [A/B test significance calculator](https://alidemirbas.com.tr/en/calculators/ab-test), and it will tell you whether the search purchase conversion gap goes beyond chance. Differences in average order value and margin are outside that calculation, so read them with the same caution.

## Frequently Asked Questions

### Should both versions use the same products and controls?

Yes. Keep the product pool for each query fixed and change only default sorting. Preserve the ability to switch sort order and keep filters unchanged.

### Should analysis focus on people reaching search results?

That can be the primary triggered population if the same trigger is logged in both groups. Assign users before they see different sorting. Do not later restrict analysis to clickers or buyers.

### Is higher conversion enough for bestseller sorting?

Use purchase conversion as the primary outcome while checking order value and gross margin. Sorting may shift the product mix. Apply consistent discount, cancellation and accounting definitions.

### What does changing the sort order tell me?

It can suggest how well the default meets expectations. Review new and returning users separately. Do not prioritize unavailable products or disguise sponsored placements as organic results.