Ali Demirbaş - Lab
TRGet in touch

Dashboard

Testing a One-Time Feature Tip Card on a Dashboard

A feature may be available in the interface without being discovered.

  • Dashboard
  • Feature trial rate

Control vs Variant

Absent on one side, present on the other.

Changed
Presence
Page
Dashboard
Difference
added

Hypothesis

One dismissible dashboard tip may draw attention and encourage first use, provided it does not interrupt the user's current task.

How to run this test

Primary metric

Feature trial rate

Among eligible users who have not used the feature before and were assigned in advance, compare first use within the defined period.

Other metrics

  • Repeat feature useCompare repeat use over the same period and report the transition from first to repeat use separately.
  • Time to complete the main taskMake sure the tip does not slow the task the user is trying to complete.
  • Tip dismissal rateUse dismissal only as a diagnostic measure within the variant.
  • Support contactsMake sure the tip does not increase confusion-related support requests.

What to check before and during the test

  • Single tipShow one predefined, dismissible card for a feature the user has not used.
  • EligibilityUse actual usage history and a fixed display trigger.
  • Keep constantKeep feature access, dashboard layout and the main task unchanged.
  • FrequencyShow the card once; after dismissal, do not show it again during this test.
  • Separate testsTest copy, timing, multiple tips and repeat display separately.

Setup mistakes to avoid

  1. 1Do not change the tip content and timing together.
  2. 2Do not show a dismissed tip again; this test is limited to one display.
  3. 3Do not let the tip obstruct the task the user is doing.
  4. 4Show no more than the one predefined tip at the defined opportunity.
  5. 5Target tips using actual usage history, not assumptions.