Dashboard
Reordering Dashboard Widgets by Recent Use A/B Test
A fixed widget order is predictable, but a widget someone needs may sit lower on the page. Bringing the most recently used widget forward may help visitors resume work, while moving controls can make them harder to find.
A/B test scenario. This page is a test plan, not the result of a real experiment.
- Dashboard
- Widget interaction rate
Control vs Variant
The sequence is the variable.
- Changed
- Order
- Page
- Dashboard
- Difference
- changed
Hypothesis
If the most recently used widget moves to the top of the dashboard, users may get back to their work without searching for it. Widget interaction rate may go up, though if moved widgets become harder to find, the time to complete the main task could grow.
Test plan
When to run this test
Look for users who work hard to reach the same widget at every sign-in: heatmaps with repeated scrolls to one widget low on the page, session recordings with long searches, or usage data on often-used widgets that sit far down. If the new order works, users reach the widget they need faster and finish their main task sooner. If the problem is widget content or slow loading, solve that first; before launch, define protected positions for critical widgets such as billing and important alerts, and add a way to turn personalization off.
How to measure success
Widget interaction rate
Compare the share of assigned dashboard users who open or use at least one widget within the predefined period.
Also track
- Time to complete the main taskCheck whether users find the information they need more quickly.
- Missing-widget complaintsTrack support contacts asking where a widget went.
- Reset-to-default rateTrack returns to the fixed order within the variant; changing a preference is not automatically a failure.
- Page load timeMake sure personalization does not slow the page.
How to set it up
- Sort orderCompare a fixed order with a predefined most-recently-used rule.
- Update frequencySet update frequency in advance; do not compare recency with frequency of use in this test.
- Keep constantKeep widget content, access and user-pinned positions unchanged.
- NavigationReview task time and difficulty finding widgets.
- DeviceReview mobile and desktop results separately.
What to watch when reading the result
- 1A rise in widget interaction can also come from users hunting for a widget that moved. If main-task time grows or missing-widget complaints increase, do not ship on this result. Read resets to the fixed order separately within the variant; that preference alone is not a failure.
- 2The result is specific to this sorting rule, this update frequency and this dashboard; keep mobile and desktop apart when you read it. Check that ordering relies only on widget usage data, infers nothing sensitive about the user and does not add to page load time.
- 3When the gap is small, allow for users adjusting to the new layout and muting the effect in the first days, and run the test for the full period you set in advance. At the end, use the A/B test significance calculator to check whether the gap stands out from sampling noise.
Frequently Asked Questions
Should automatic sorting move pinned widgets?
No. Preserve explicit pinned positions, widget access and contents. Give users a way to disable personalization rather than overriding their choices.
Does more widget interaction mean tasks are easier?
Not necessarily. Use the defined widget interaction outcome, then inspect task time and reports of missing widgets. Additional searching can increase interactions without improving usability.
How frequently should the order update?
Choose the update frequency before launch and hold it stable. Comparing recent use against frequent use is a different sorting question.
What happens to rarely used but critical widgets?
Define protected positions or access rules first. Apply usage-based ordering within those constraints. Low usage does not establish low importance, especially for billing and urgent alerts.
More tests in Dashboard
All testsTesting a One-Time Feature Tip Card on a Dashboard
DashboardFeature trial rate
A little-known feature introduced with one dismissible tip card, versus no introduction. Includes the interruption risk and the feature trial metric.
Empty-State Action Card on a Dashboard A/B Test
DashboardFirst-action completion rate
A new user opens a dashboard with no data yet. Compare a bare empty table with a card pointing to a first action, tracked as first-action completion.
Above vs Left-Aligned Form Label A/B Test Example
Forms & signupForm completion rate
Labels above the input against labels aligned to its left. Read why scan path, vertical space and long or translated labels pull the decision both ways.