Category & Listing
Menu Category Order A/B Test on Listing Pages
Categories near the top of a menu may draw more attention. More clicks on those categories do not necessarily mean more purchases; use conversion as the decision metric and browsing measures to explain the result.
A/B test scenario. This page is a test plan, not the result of a real experiment.
- Category listing
- Purchase conversion rate (CR)
Option A vs Option B
Position in the menu is the variable.
- Changed
- Menu order
- Page
- Category listing
- Difference
- changed
Hypothesis
An order based on category-page visits may move the sections shoppers enter most to the top and make them quicker to reach, but it may also push down high-revenue categories that get fewer visits. Whether the visit-based order or the current one delivers the higher purchase conversion rate may depend on whether easier access outweighs that loss.
Test plan
When to run this test
Analytics showing heavy traffic to categories listed low in the menu, session recordings of visitors browsing the menu at length, or site searches for category names the menu already contains suggest that shoppers struggle to reach the category they want. If the visit-based order helps, shoppers reach their category with fewer interactions and that easier access shows up in purchases. If shoppers misread the category names themselves, review the names first, because reordering does not fix an unclear label.
How to measure success
Purchase conversion rate (CR)
Compare purchases among all eligible visitors assigned to each menu order, not only those who open the menu or select a category.
Also track
- Category click-through rateCompare clicks on categories moved higher in the menu.
- Menu interaction depthCount interactions needed to reach a category; this does not measure elapsed time.
- Menu scroll depthUse scroll behavior to describe navigation, not as proof that content was read.
- Exit rateTrack whether visitors leave more often after seeing the menu.
How to set it up
- OrderCompare the current menu order with the same categories ordered by category-page visits over the 90 days before launch.
- Keep constantKeep category names, icons and menu structure unchanged.
- NavigationMeasure category clicks and the path to the selected category.
- ConversionCompare purchases across all eligible assigned visitors, not only category clickers.
- DeviceReview mobile and desktop menus separately.
What to watch when reading the result
- 1More clicks on a category moved to the top do not make that order the winner. Judge purchase conversion across all eligible assigned visitors, not only menu clickers; if high-revenue categories lose sales, an order built on visits alone should not go live.
- 2Make sure high-value categories stay findable in both orders. Read the mobile and desktop menus apart; if the categories moved up crowd the mobile menu or menu interaction depth rises, do not ship that order on mobile. The finding only speaks for this category set and an order built from this period's visit data.
- 3If the gap between the two orders stays small, first confirm that every visitor saw the menu order assigned to their arm for the whole test. Also check that each arm received the share of visitors you planned (sample ratio mismatch); if the split is off, do not read the difference. If the split is clean, see whether the purchase gap is significant with the A/B test significance calculator; if it is not, record that the test could not tell the two orders apart.
Frequently Asked Questions
Are more clicks on top menu categories sufficient?
No. Use purchase conversion as the primary outcome. Include all assigned eligible visitors, not just menu clickers, and examine whether the ordering creates additional purchases.
Does reduced click depth mean faster category discovery?
Click depth measures interaction count, not elapsed time. Measure time to the category separately using consistent start and destination events.
What is the risk of ranking menu items only by visits?
Important revenue-generating categories may become harder to find. Preserve their accessibility and avoid crowding the mobile menu with promoted items.
More tests in Category & Listing
All testsShort vs Descriptive Category Headings A/B Test
Category listingCategory-page purchase conversion rate
An experiment idea comparing a short category heading with a descriptive one on the listing page. Look at both arms, the attention reasoning and the metric.
Pop-Up Timing on Category Pages: A/B Test Idea
Category listingPurchase conversion rate
An open-ended experiment idea about when a pop-up should appear. See the tested variable and the reasoning on pop-ups that fire too early or too late.
How Many Products Per Page? A/B Test Example
Category listingProduct detail page visit rate
An open-ended experiment idea on listing density. See the trade-off described between fewer page loads and easier comparison versus heavier, slower pages.