# A/B Test Sample Size Calculator

Page: https://alidemirbas.com.tr/en/calculators/sample-size-calculator

Enter your baseline conversion rate, the smallest lift worth detecting, power and confidence level to get the sample size each variant needs.

**Sample size formula with example**

`n per variant = [z₁₋α/₂ × √(2 × p̄ × (1 − p̄)) + z₁₋β × √(p₁ × (1 − p₁) + p₂ × (1 − p₂))]² ÷ (p₂ − p₁)²`

1. Turn the relative MDE into the variant's expected rate: p₂ = p₁ × (1 + MDE). A 10% MDE on a 5% baseline means detecting a move from p₁ = 5.0% to p₂ = 5.5%, an absolute effect of 0.5 points.
2. Pick the z-values: 1.645, 1.96 or 2.576 for 90%, 95% or 99% confidence (two-sided), and 0.8416 or 1.2816 for 80% or 90% power.
3. Weight the confidence z-value by the variance of the average rate p̄ = (p₁ + p₂) ÷ 2 under "no difference", and the power z-value by the two groups' own variances. Add the two terms, square the sum, divide by (p₂ − p₁)² and round up to the next whole visitor.

`[1.96 × √(2 × 0.0525 × 0.9475) + 0.8416 × √(0.05 × 0.95 + 0.055 × 0.945)]² ÷ 0.005² = (0.6182 + 0.2654)² ÷ 0.000025 = 31,233.77 → 31,234 visitors per variant`

## What is A/B test sample size

A/B test sample size is the number of visitors each variant needs before the test has a fair chance of detecting a lift of the size you care about. This sample size calculator works backward from four inputs: your baseline conversion rate, the minimum detectable effect (MDE), statistical power and the confidence level. With too few visitors a real improvement can pass unnoticed, because the data cannot separate it from noise.

The calculation is the standard power formula for comparing two proportions. The [NIST/SEMATECH e-Handbook of Statistical Methods](https://www.itl.nist.gov/div898/handbook/prc/section2/prc222.htm) gives the sample size needed to detect a shift δ in a mean as (z₁₋α/₂ + z₁₋β)² × (σ/δ)². For two conversion rates, each group has its own variance p × (1 − p): this calculator uses the average rate's variance for the confidence term and the control and variant variances for the power term, the form common two-proportion calculators use. A shortcut that plugs the baseline variance into both groups returns a smaller number, about 5% smaller at a 10% relative MDE and more for larger lifts, and leaves the test below the power you chose.

MDE in this calculator is relative. Entering 10 means detecting a 10% relative improvement: from a 5% baseline that is a move to 5.5%, a change of only half a percentage point. At a 2% baseline it means about 2.2%, at 40% about 44%. The relative vs absolute MDE convention can explain why two tools disagree. If another calculator gives a very different number for the same inputs, check how it treats MDE before assuming either is wrong.

> Decide the sample size before the test starts and run to it, because stopping when the numbers look good turns it into a different experiment.

## Sample size per variant by baseline rate and MDE

The table shows how the two inputs you control most directly drive the number. Every cell uses the formula above with 95% confidence and 80% power and shows visitors per variant. Read across a row to see the cost of chasing smaller lifts, and down a column to see how a low baseline conversion rate raises the bar.

| Baseline conversion rate | MDE 5% (relative) | MDE 10% (relative) | MDE 20% (relative) |
| --- | --- | --- | --- |
| 1% | 637,017 | 163,097 | 42,694 |
| 2% | 315,210 | 80,683 | 21,109 |
| 5% | 122,125 | 31,234 | 8,158 |
| 10% | 57,764 | 14,751 | 3,841 |
| 20% | 25,583 | 6,510 | 1,683 |

Halving the MDE roughly quadruples the sample, because the required sample size scales with one over the square of the effect. A low baseline works against you in the same way. On the 5% baseline, 10% MDE example, raising power from 80% to 90% takes the requirement from 31,234 to 41,815 per variant. Moving from 95% to 99% confidence takes it to 46,480. Base the MDE on the smallest lift that would change your decision. An MDE picked only to make the number look affordable leaves the test underpowered for smaller lifts that would still matter.

## From sample size to test duration

A sample size becomes a plan once you turn it into calendar time. Test duration follows from three numbers:

- Total sample: multiply the per-variant number by the number of variants. A two-variant test at 31,234 per variant needs 62,468 visitors in total.
- Daily eligible traffic: count only visitors who actually reach the page or step being tested. Total site traffic overstates it.
- Full weeks: divide the total by daily traffic and round up to whole weeks, so weekday and weekend behavior are both represented. At 4,000 eligible visitors a day the example needs 16 days, which rounds up to 21.

If the duration comes out at several months, change the plan before launch. Test a bolder change with a larger expected effect, raise the MDE, settle for 80% power or move the test to a busier step of the funnel. The [multi-step funnel analysis](https://alidemirbas.com.tr/en/calculators/funnel-analysis-multistep) shows where traffic and drop-off are largest. When the test ends, read the result once with the [A/B test significance calculator](https://alidemirbas.com.tr/en/calculators/ab-test). The [complete guide to A/B testing](https://alidemirbas.com.tr/en/blog/what-is-ab-testing-complete-guide) covers the rest of the process, from hypothesis to rollout.

## When to use the sample size calculator

Use it before launch, to check whether a test is feasible on your traffic and to fix the stopping point in advance. Take the baseline from the same page or step and the same audience the test will run on, over a normal period. A baseline taken from a seasonal peak or a single campaign makes the estimate unreliable. You can measure it with the [conversion rate calculator](https://alidemirbas.com.tr/en/calculators/cr).

## Where a sample size estimate misleads

The estimate is only as good as the baseline behind it. If the real rate during the test is lower than the one you entered, the test ends underpowered. The formula also assumes a 50/50 split and independent visitors, so uneven traffic allocation or returning visitors counted twice need a larger sample than the number shown. A sample size calculated for one primary metric does not cover secondary metrics or segments, which usually have far less data.

## Frequently Asked Questions

### How many visitors do I need for an A/B test?

It depends on your baseline conversion rate, the smallest lift you want to detect, the power and the confidence level. With a 5% baseline, a 10% relative MDE, 80% power and 95% confidence, the formula gives 31,234 visitors per variant, or 62,468 for a two-variant test. A higher baseline or a larger MDE lowers the number, and a smaller MDE raises it sharply. There is no universal minimum that fits every test, so calculate it for your own page before launch.

### What is the minimum detectable effect (MDE)?

The minimum detectable effect is the smallest true lift the test is designed to catch with the chosen power. In this calculator it is relative: an MDE of 10% on a 5% baseline means a change to 5.5%. Choose it from the business side, as the smallest improvement that would justify shipping the change. A smaller MDE needs much more traffic, since halving it roughly quadruples the sample. Effects smaller than the MDE can still be real, but the test will often miss them.

### Should I use 80% or 90% power?

Use 90% when missing a real effect is expensive, for example before a costly rebuild, and 80% for routine tests on pages with limited traffic. 80% power is the common default: if the true lift equals your MDE, the test detects it four times out of five. 90% power cuts the miss rate to one in ten but needs more visitors. In the 5% baseline, 10% MDE example the requirement rises from 31,234 to 41,815 per variant, about a third more.

### How long should an A/B test run?

Divide the total sample (per-variant sample × number of variants) by the daily traffic that reaches the tested page, then round up to whole weeks. At 4,000 eligible visitors a day, a two-variant test needing 31,234 per variant takes about 16 days, so plan for three full weeks. Running in whole weeks keeps weekday and weekend behavior in both groups. Set the end date before launch and do not stop early when the result looks good, because that raises the false-positive rate.

### What if I don't have enough traffic to reach this sample size?

You have four levers. Raise the MDE and accept that only larger lifts will be detectable. Test a bolder change that could plausibly produce that larger lift. Move the test to a higher-traffic page or an earlier funnel step. Or lower power or confidence and accept a higher chance of missing a real effect or seeing a false positive. Extending the test also helps, though very long tests pick up seasonal shifts in the audience. Avoid running with fewer visitors than required and reading the result anyway.