How to Actually Read Your A/B Test and Ad Campaign Numbers (Not Just Eyeball Them)
Published October 20, 2026
Conversion rate, statistical significance, CTR, and ROAS each answer a different question about a campaign — here's what each one actually measures, how they relate, and why the biggest number in a dashboard isn't automatically the winning one.
A dashboard full of percentages is easy to misread with confidence. Variant B converted at 4.2% and variant A converted at 3.6% — ship B, right? Maybe. Or maybe that's 14 conversions out of 390 visitors versus 11 out of 305, a gap small enough that it could just as easily be noise. Marketing numbers aren't hard to calculate, but they're surprisingly easy to misinterpret, because several different metrics all look similar (they're all percentages, they're all "performance") while actually answering different questions. Here's how to read the four metrics that come up constantly, and more importantly, how they relate to each other.
Conversion rate: the baseline everything else builds on
Conversion rate is the simplest of the four and the foundation for the rest: of the people who did X (visited a page, saw an ad, started a signup flow), what percentage completed the goal action (bought something, submitted a form, created an account)? Conversion Rate Calculator does the division for you, but the number itself only becomes meaningful once you know what "X" was — conversion rate from ad click to purchase and conversion rate from landing page visit to purchase can be wildly different numbers describing different parts of the same funnel, and conflating them is a common source of confused comparisons between campaigns that are actually measuring different stages.
A realistic example: an email campaign sends 2,000 people to a signup page, and 86 of them complete signup. That's a 4.3% conversion rate. On its own, is that good? It depends entirely on context — industry, offer, traffic quality, and what you're comparing it against. Which is exactly why conversion rate is rarely useful in isolation; it's useful as one side of a comparison, which brings up the next, much more commonly misused, metric.
Why you can't just eyeball two conversion rates and call a winner
This is where a lot of A/B testing goes wrong, and it's worth being precise about why, because the oversimplified version ("just run it longer") and the overcomplicated version ("you need a statistics degree") are both unhelpful.
Here's the actual issue: conversion rate is calculated from a sample of visitors, not from every person who will ever see the page. Any sample has random variation built into it, purely from chance — if you flipped two fair coins 50 times each, you wouldn't be shocked if one came up heads 27 times and the other 23, even though both coins are identical. Small differences between two groups can happen by pure chance even when there's no real underlying difference between the two things being tested. The smaller the sample, the more a result can bounce around just from random luck, which is exactly why a difference that looks real in a 50-visitor test often evaporates once you collect 5,000 visitors.
Statistical significance testing is the formal way of asking: "given how much random variation we'd expect from samples this size, how surprising is the gap we actually observed?" A/B Test Significance Calculator takes the visitor count and conversion count for both variants and calculates this for you, typically expressed as a p-value or a confidence level. A common threshold is 95% confidence — meaning if there were truly no difference between the two variants, you'd expect to see a gap this large (or larger) only 5% of the time by chance alone.
A few things worth being precise about here, because this is where people oversimplify in ways that actually matter:
- Reaching significance doesn't mean "guaranteed to keep winning forever." It means the observed gap is unlikely to be pure chance given the data collected so far. Underlying conditions change — seasonality, traffic sources, competitor activity — so a variant that wins decisively in March isn't guaranteed to still be the better choice in September.
- Not reaching significance doesn't mean "they're equal." It often just means you don't have enough data yet to tell the difference from noise. The honest conclusion from an inconclusive test is "we don't know," not "it doesn't matter which one we use."
- Checking results early and stopping the moment they look significant is a real statistical mistake (often called "peeking"), because the more times you check a noisy metric, the more likely you are to catch it at a moment it randomly looks significant even with no real effect underlying it. Decide on a sample size or time window in advance where practical, and be skeptical of a result that flipped significant on day two of a planned two-week test.
- A statistically significant result is not automatically a practically significant one. A rigorously confirmed 0.1 percentage point lift might not be worth the engineering effort to ship, even though the math behind it is sound.
None of this means significance testing isn't useful — it's exactly the tool for separating "there's probably a real effect here" from "we're reacting to noise." It just means the output is a statement about probability and confidence, not a guarantee, and it's worth reading it that way rather than as a verdict.
CTR measures the hook, not the outcome
Click-through rate — clicks divided by impressions — answers a narrower and earlier question than conversion rate: did the ad or headline get someone's attention enough to click? CTR Calculator turns raw impression and click counts into that percentage.
The reason it's worth tracking separately from conversion rate is that the two can move in opposite directions, and that divergence is itself informative. An ad with a dramatic, slightly misleading headline might pull a high CTR by getting lots of curious clicks — and then a low conversion rate, because the people who clicked weren't actually the right audience or weren't shown what they expected once they landed. Conversely, a plain, highly specific headline might get fewer clicks overall but convert much better among the people who do click, because it pre-qualified the audience before they ever landed on the page. Neither pattern is automatically wrong — it depends on whether the goal is top-of-funnel awareness (where raw clicks matter more) or bottom-of-funnel conversion (where qualified clicks matter more) — but you can't tell which situation you're in by looking at either metric alone. CTR tells you about the hook; conversion rate tells you about everything that happens after the click.
ROAS: the metric that ties it back to money
Return on ad spend answers the question the other three don't: for every dollar spent on this campaign, how many dollars came back in revenue? ROAS Calculator divides revenue generated by ad spend, usually expressed as a ratio like 4:1 or simply "4x."
This is the metric that keeps the other three honest, because a campaign can look great on CTR and conversion rate while still losing money, if the cost per click is high enough or the average order value is low enough. A campaign converting at a strong 6% with an excellent CTR can still have a ROAS under 1x — meaning it's spending more than it brings back — if the targeting is pulling in expensive clicks against a low-margin product. Conversely, a campaign with a mediocre conversion rate can still post a healthy ROAS if the cost per click is low and the average transaction value is high.
Worth being careful about: ROAS looks at revenue, not profit. A 4x ROAS sounds strong, but if the product's margin is thin, 4x in revenue might only be a small amount in actual profit once cost of goods is factored in — ROAS alone doesn't account for that, so comparing ROAS across products or campaigns with very different margins can be misleading without also considering margin.
Reading all four together: a realistic scenario
Say you're comparing two ad variants for the same product over a two-week test:
- Variant A: 40,000 impressions, 1,200 clicks, 54 conversions, $3,100 spend, $8,900 revenue.
- Variant B: 38,500 impressions, 1,650 clicks, 46 conversions, $3,050 spend, $7,400 revenue.
Run the clicks and impressions through CTR Calculator and B has the noticeably higher CTR — its creative or headline is pulling more clicks per impression. Run the conversions against the respective click counts through Conversion Rate Calculator and the picture flips: A is converting a meaningfully higher share of the people who actually click. Feed both variants' visitor and conversion counts into A/B Test Significance Calculator to see whether that conversion rate gap is large enough, given the sample sizes involved, to trust as a real effect rather than noise — that's the step that tells you whether you're looking at a real signal or something that could easily flip with another two weeks of data. And running spend against revenue for each through ROAS Calculator shows A returning meaningfully more per dollar spent despite the lower click volume, because the clicks it does get convert at a higher rate.
The takeaway isn't "always trust conversion rate over CTR" — it's that each of these four numbers is answering a different, narrower question, and a campaign that wins on one can lose on another. Reading them together, with significance testing as the gate on whether a gap is worth trusting at all, is what turns a dashboard of percentages into an actual decision.
FAQ
Which metric matters most? It depends on the goal. For pure awareness or top-of-funnel campaigns, CTR matters more. For anything tied to revenue, ROAS is the one that ultimately decides whether to keep spending.
How much traffic do I need before a significance test means anything? There's no universal number — it depends on the baseline conversion rate and how big a difference you're trying to detect. Smaller expected differences require larger samples to detect reliably. If a significance calculator says you're far from a conclusive result, that's useful information on its own, not a reason to ignore the test.
Can a test be significant but still not worth acting on? Yes — a confirmed but tiny lift might not justify the cost of implementing or maintaining the winning variant. Significance tells you the effect is probably real; it doesn't tell you the effect is big enough to matter.
Is a high ROAS always good? Only relative to margin and business goals. A high ROAS on a loss-leader product funding customer acquisition can be a deliberate, reasonable strategy, not a mistake — context matters more than the raw ratio.