Ali Demirbaş - Lab
TRGet in touch

Dashboard

Reordering Dashboard Widgets by Recent Use A/B Test

A fixed widget order is predictable, but a widget someone needs may sit lower on the page.

  • Dashboard
  • Widget interaction rate

Control vs Variant

The sequence is the variable.

Changed
Order
Page
Dashboard
Difference
changed

Hypothesis

Bringing the most recently used widget forward may help visitors resume work, while moving controls can make them harder to find.

How to run this test

Primary metric

Widget interaction rate

Compare the share of assigned dashboard users who open or use at least one widget within the predefined period.

Other metrics

  • Time to complete the main taskCheck whether users find the information they need more quickly.
  • Missing-widget complaintsTrack support contacts asking where a widget went.
  • Reset-to-default rateTrack returns to the fixed order within the variant; changing a preference is not automatically a failure.
  • Page load timeMake sure personalization does not slow the page.

What to check before and during the test

  • Sort orderCompare a fixed order with a predefined most-recently-used rule.
  • Update frequencySet update frequency in advance; do not compare recency with frequency of use in this test.
  • Keep constantKeep widget content, access and user-pinned positions unchanged.
  • NavigationReview task time and difficulty finding widgets.
  • DeviceReview mobile and desktop results separately.

Setup mistakes to avoid

  1. 1Do not change sorting rule and update frequency together.
  2. 2Do not move a widget the user pinned using automatic sorting.
  3. 3Do not hide critical billing or warning widgets because they are used less often.
  4. 4Base sorting on usage data; do not infer sensitive information.
  5. 5Provide a way to turn personalization off.