# Testing a One-Time Feature Tip Card on a Dashboard

Page: https://alidemirbas.com.tr/en/lab/ab-test-library/tests/testing-a-one-time-feature-tip-card-on-a-dashboard

## Hypothesis

If a user who has never used the feature sees one dismissible tip on the dashboard, they may notice the feature exists and give it a try. More eligible users may then try it within the defined period, lifting the feature trial rate.

## When to run this test

This test makes sense when a valuable feature is available but seems undiscovered, such as accounts that never open it, support tickets asking for something the product already does, or survey answers from users who never heard of it. If the tip works, users who see it try the feature more often and come back to it after the first try. If people find the feature but it does not help them, a tip will not fix that, so work on the feature first; before launch, also confirm the card does not get in the way of the task the user is doing.

## Primary metric

**Feature trial rate**: Among eligible users who have not used the feature before and were assigned in advance, compare first use within the defined period.

## Other metrics

- **Repeat feature use**: Compare repeat use over the same period and report the transition from first to repeat use separately.
- **Time to complete the main task**: Make sure the tip does not slow the task the user is trying to complete.
- **Tip dismissal rate**: Use dismissal only as a diagnostic measure within the variant.
- **Support contacts**: Make sure the tip does not increase confusion-related support requests.

## What to check before and during the test

- **Single tip**: Show one predefined, dismissible card for a feature the user has not used.
- **Eligibility**: Use actual usage history and a fixed display trigger.
- **Keep constant**: Keep feature access, dashboard layout and the main task unchanged.
- **Frequency**: Show the card once; after dismissal, do not show it again during this test.
- **Separate tests**: Test copy, timing, multiple tips and repeat display separately.

## What to watch when reading the result

- The feature trial rate is the primary measure, but if main-task time grows or support contacts rise, more first uses do not justify shipping the tip. That pattern may mean the tip gets in the way of the user's real work; read dismissal only as a diagnostic within the variant.
- A first try does not show adoption on its own. Compare repeat feature use among all assigned users over the same period, report the move from first to repeat use separately, and treat the result as valid only for this feature and this tip.
- Because only users who have never used the feature qualify, the eligible audience may be small. Work out how many users you need to detect the difference you expect with the [sample size calculator](https://alidemirbas.com.tr/en/calculators/sample-size-calculator), and if the result is unclear, check whether you reached that number before extending the test.

## Frequently Asked Questions

### Who should see the feature tip?

Users who have not used the feature and meet the predefined eligibility rule. Base targeting on recorded history and a fixed display trigger rather than assumed behavior.

### Should the tip return after dismissal?

Not in this one-time-tip scenario. Keep the single display opportunity fixed. Repetition and multiple tips are separate comparisons.

### Does more first-time use prove adoption?

No. Use first feature use as the primary outcome and report repeat use over equal windows. Analyze all assigned eligible users rather than just tip clickers.

### How could the tip interfere with work?

An irrelevant interruption can delay the user’s main task. Monitor task completion time and related support contacts, and avoid blocking the underlying workflow.