Notification Badge: Number or Dot A/B Test Example

Low section visits while the badge is showing, user research where people ask what the badge means, or support tickets about pending items that went unnoticed mean the notification badge is either missed or misunderstood. If the count clarifies the work, users who see it enter the section and complete the pending task; if it makes the work feel larger, visits run higher with the dot. Before launch, confirm the badge appears only for real pending items and that users can clear it by reading or dismissing them.

Visit rate for the indicated section

Compare visits to the section indicated by the badge among eligible assigned users.

Also track

• Action completion rateCompare completion of the predefined task in the indicated section.
• Badge interaction rateReport visits to the indicated section; a visit alone does not prove the task was completed.
• App or site exit rateCompare exits within the same session window.

• FormatCompare a numeric badge with a dot for the same pending state.
• Count rangesReview predefined ranges based on actual pending-item counts.
• ClearingKeep the same read or clear behavior.
• PlacementKeep the same control and position.
• DeviceCheck readability and discoverability on mobile.

1. 1Do not call either format a winner just because the visit rate or the badge interaction rate is higher; tapping a badge does not mean the task got done. Check whether the difference carries through to the action completion rate in the indicated section, and confirm the app or site exit rate does not rise.
2. 2Read the result separately for the predefined ranges of real pending counts, since a few items and many items may draw different reactions. Check that both formats can be noticed without relying on red color alone, and review readability on mobile screens. Do not carry the result over to badges on other controls or positions without testing them there.
3. 3Once you split the result by count range, each range on its own is a small group, so differences there can easily stay unclear. Set the number of users per arm that each range needs before launch with the sample size calculator.