All testsAli Demirbaş - Lab
TRGet in touch

Search & Filtering

Filter Checkboxes vs a Guided Question Flow A/B Test

Questions about needs or budget may help shoppers who are unsure which filters to choose. Visitors who already know what they want may find the extra steps slow. Offer one defined guided path while preserving direct access to the same underlying filters.

A/B test scenario. This page is a test plan, not the result of a real experiment.

  • Filters
  • Product page visit rate

Control vs Variant

The same value, presented two ways.

Changed
Format
Page
Filters
Difference
changed

Hypothesis

If filtering is guided by questions such as who the product is for and what the budget is, visitors who are unsure which options to tick may reach suitable products more easily. Product page visit rate may go up as a result, while visitors who already know what they want may be slowed by the extra steps.

Test plan

Calculate sample size

When to run this test

Filters selected and removed right away, combinations that often end in zero results, or support requests asking which product is right all show visitors struggling to decide what to tick in the filter panel. If the guided flow helps, its users reach a product page with less trial and error, and those who filter directly are not slowed down. If the cause is confusing filter labels or missing product attributes, fix those first, because a question flow cannot repair faulty product data.

How to measure success

Product page visit rate

Compare product-page visits per eligible assigned visitor while keeping the classic filter panel available in both arms.

Also track

  • Guided-flow completion rateReport completion among visitors who start the guided flow; keep the primary analysis on all assigned visitors.
  • Classic filter usageTrack use of the classic filters, which remain available in both arms.
  • Zero-result rateCompare eligible searches that end with no results.
  • Time to reach a productMeasure time using the same start and arrival events; report visitors who never arrive.

How to set it up

  • PathCompare the classic filter panel with one predefined guided question flow that has the classic panel as a fallback.
  • SkipKeep direct access to filters available before the questions are completed.
  • QuestionsFix the number, order and wording of questions before launch.
  • MappingMap equivalent answers to the same core attributes and result rules.
  • Visitor historyReview new and returning visitors separately.

What to watch when reading the result

  1. 1A higher guided-flow completion rate does not pick the winner; report it only among visitors who start the flow, as a diagnostic. The deciding metric is product page visit rate across all eligible assigned visitors; when you report time to reach a product, list visitors who never get there separately.
  2. 2Review the answer combinations that end with no results; each of those screens needs a way to edit answers or switch to the classic filters. A dead end drags down the variant's result even if the idea of a guided flow works.
  3. 3Leave answers out of the next visit in this test; keeping them without asking the user is a separate change. Read new and returning visitors separately, and treat the result as valid only for this question set and these mapping rules.
  4. 4An unclear difference may mean the effect is diluted, because only some assigned visitors use the flow. Recalculate the visitors needed per variant by entering the product page visit rate for everyone assigned and that diluted lift into the sample size calculator. Dividing the result by each arm's daily traffic shows how much longer the test has to run.

Frequently Asked Questions

Can the guided questions be mandatory?

No. Preserve direct filtering before completion and access to the classic panel in both groups. Equivalent answers must use the same filtering rules. Provide editing and exit routes for empty combinations.

Can guided-flow completion determine the winner?

No. Report completion among flow starters diagnostically. Use product-page reach rate as the primary outcome.

Can answers be reused on the next visit?

Not in this test. Persistence is a separate change requiring a defined purpose, retention period and user controls. Review new and returning visitors separately.

More tests in Search & Filtering

All tests

Applying Filters Instantly or With a Button A/B Test

FiltersSuccessful product discovery after filtering

A CRO experiment on how filters are applied: results updating immediately against an apply button that submits selections together. Both arms and the KPI.

Search & Filtering

Header Search Field vs Menu Search A/B Test

SearchPurchase conversion rate (CR)

Compare an always-visible search field in the header with search placed inside the menu, tracking search use and downstream conversion.

Search & Filtering

Previous Searches vs Empty Search Box A/B Test

SearchPurchase conversion rate (CR)

Compare showing previous searches when the search box opens with an empty search box, tracking search use, conversion and privacy concerns.

Search & Filtering