A/B Test Significance Calculator

Check whether your A/B test results are statistically significant using conversion rates and sample sizes from each variant. This free online calculator helps you decide whether to roll out a new headline, button, or pricing page design, with instant results and no sign-up required.

Control (A)
Variant (B)

—

Usage examples

Deciding whether to roll out a new variant

Check whether your B variant's higher conversion rate is a real improvement or just random noise before rolling it out to everyone.

Checking if you can stop a test early

See whether your current results have already reached statistical significance, or whether you need to keep collecting data.

Comparing two headline or button designs

Enter the visitor and conversion counts for each version to see which one actually performed better, not just which number looks bigger.

Evaluating a pricing page experiment

Enter visitor and conversion counts for two pricing page layouts to see how the data breaks down before deciding which one to keep.

Reviewing a past test before documenting results

Re-enter the final visitor and conversion numbers from a completed test to double-check the p-value and confidence levels before writing up the results.

Frequently asked questions

What does 'statistically significant' actually mean?

It means the difference you observed is unlikely to be due to random chance alone, at the confidence level you chose. A p-value below 0.05 is the most common threshold for '95% confidence'. It does NOT mean the difference is large, important, or guaranteed to hold up forever - just that it's unlikely to be pure noise given your sample size.

Can a 'significant' result still be wrong?

Yes. At 95% confidence, there's still roughly a 1-in-20 chance of a false positive by definition. Significance also doesn't account for running many tests at once, or stopping a test as soon as it looks good - both of which inflate the real false-positive rate beyond what the raw p-value suggests.

My sample size is small - can I trust this?

Small samples make it much easier to see a 'significant' result that doesn't hold up with more data, and much harder to detect a real but modest difference. Treat results from small sample sizes cautiously and consider running the test longer.

Is my input sent to a server?

No. All calculations happen entirely in your browser and are never sent to a server.

What's the difference between absolute lift and relative lift?

Absolute lift is the straightforward difference between the two conversion rates (e.g. 12% minus 10% is a 2 percentage point absolute lift). Relative lift expresses that same difference as a percentage of the original rate (2 percentage points on a 10% baseline is a 20% relative lift). Headlines often quote relative lift because the number looks bigger, so check which one is being reported before drawing conclusions.

What does the Z-score actually represent?

The Z-score measures how many standard deviations the observed difference between your control and variant is from zero (no difference), given the sample sizes involved. A larger Z-score (in either direction) generally corresponds to a smaller p-value and stronger evidence of a real difference, assuming the test's statistical assumptions hold.

Does this tool tell me how long to keep running my test?

No. This calculator only evaluates the data you've entered so far — it doesn't calculate a required sample size or a minimum test duration ahead of time. For planning a test, you'd typically want a separate sample-size calculator that factors in your baseline rate, minimum detectable effect, and desired power.

Is my data sent to a server?

No. All calculations happen entirely in your browser and are never sent to a server.