Conversion rate: how to calculate it, and what to do with the result
The formula, the right denominator, rates for sales, leads and ads, and why comparing yourself with an average is a trap.
A good A/B testing process compares two versions of the same page, shown at random to different visitors, based on a hypothesis written before launch. You decide the metric, the sample size and the duration in advance, run the test over complete weeks, resist stopping it when the numbers look good, and make the call only once the planned sample has been reached.
What an A/B test is, and what it is not
In an A/B test, half of your visitors see the current version (the control, A) and the other half see a modified version (B). The split is random, and you then compare one precise metric, for example the percentage of visitors who submit a form.
What an A/B test is not: changing the page on Monday and comparing it with last month. Between those two months the season, the campaigns and the traffic have changed, and perhaps the competition too. An A/B test runs the versions in parallel precisely so that these differences affect both in the same way.
Testing is not a tool for inspiration; it is a tool for decisions. It answers the question: “does this change make a real difference, or could the variation I see be down to chance?” If you want the basic arithmetic of conversion rates, it is explained in our article on how to calculate your conversion rate.
When testing makes sense, and when it does not
A/B testing only works with enough traffic and a reasonable number of conversions. If a page gets a few hundred visitors a month and has few conversions, a classic test can run for months or years without a clear result.
- It makes sense when you have steady traffic, a page with a clear job (a product page, a form, a landing page) and a metric that is easy to measure.
- It does not make sense when traffic is very low, when the page has obvious faults (a broken form, a phone layout that is unusable) or when you already know what needs fixing.
With low traffic, you will get more from fixing the obvious problems using qualitative research: session recordings, click maps, short surveys, conversations with customers. Then you apply the changes and compare before and after, knowing that this is not perfect evidence but is still a reasonable decision. For the structure of a conversion page, see also our article on landing page structure.
How to write a good hypothesis
A test hypothesis has three parts: what you change, what effect you expect and why. A simple formula: “Because [observation], if we change [element] to [variant], we will see [effect] in [metric], since [reason].”
A hypothetical example: imagine a clinic that gets many visits to its booking page but few submitted forms. Session recordings show visitors stopping at the “reason for visit” field, which is compulsory and open-ended. The hypothesis: “Because many people abandon at the free-text field, if we turn it into a list of options, the form submission rate will rise, since the perceived effort drops.”
The difference from “let’s try a green button”: the hypothesis rests on an observation, and you can learn something even if the test loses. If the variant does not win, you find out that the reason for abandonment was something else and you can move on to a new hypothesis.
What you can test, in order of likely impact
| Element | Examples | Potential impact |
|---|---|---|
| Value proposition | main headline, the promise of the offer, subheading | High |
| The offer | guarantee, free delivery, trial period, displayed price | High |
| The form | number of fields, order, error messages | High |
| Social proof | genuine reviews, placement and size, client logos with permission | Medium |
| Call to action | wording, position, number of buttons | Medium |
| Images and video | real photography versus stock, a demo video | Medium |
| Visual details | button colour, small alignment tweaks | Low |
Small changes rarely produce visible differences and need enormous samples to be proven. That is why, in the first stage, you should test big ideas: message, offer, structure.
Sample size: why you cannot guess it
Sample size depends on three things: your current conversion rate, the smallest improvement you want to be able to detect, and the confidence level you want (usually 95%, with a statistical power of 80%). The smaller the difference you are looking for, the more visitors you need.
The figures below are indicative calculations, made with the classic formula for comparing two proportions at 95% confidence and 80% power:
| Current rate | Relative uplift sought | New rate | Visitors needed per variant (approx.) |
|---|---|---|---|
| 3% | +20% | 3.6% | about 14,000 |
| 3% | +10% | 3.3% | about 53,000 |
| 1% | +20% | 1.2% | about 43,000 |
Notice how the requirement jumps when you want to detect a small difference or when the baseline rate is low. To run the numbers on your own figures, use our conversion calculator. If the result tells you that you need several months of traffic, treat that as information: either test a bigger change, or do not run the test.
Test duration and statistical significance
Duration is not decided by how the chart looks, but by the calculated sample and the behaviour cycles of your audience. A few rules:
- Run the test over whole weeks, because weekends and weekdays behave differently.
- Where possible, cover at least two complete cycles (for instance two weeks) to reduce the effect of chance.
- Avoid unusual periods: holidays, big campaigns, launches, technical faults.
- Do not extend the test indefinitely in the hope that it will “clear up”: fix the stopping date or sample in advance.
“Statistical significance” tells you how unlikely it would be to see a difference at least this large if, in reality, the versions were equal. A result that is “significant at 95%” does not mean “a 95% chance that B is better”. It means that if there were no difference, you would see a result this extreme in only about 5% of cases or fewer. Many people misread this, which is why it is worth treating the threshold as a decision rule set in advance, not as a guarantee.
How to run a test, step by step
- Check your measurement. Without correct events there is no point testing; start from conversion tracking.
- Choose one primary metric (for example submitted forms or orders) and a few guardrail metrics (bounce rate, average order value).
- Write the hypothesis and note why it should work.
- Calculate the sample and the estimated duration; if it is not realistic, rethink the test.
- Build the variant by changing one thing, or one coherent group of things that you can explain.
- Check it technically: the variants load correctly on mobile and desktop, without a flash of the original page, and events are recorded in both.
- Start the test and leave it alone. Do not change the traffic, the copy or the budget midway.
- Stop at the planned sample, analyse the primary result, then the segments, treating those as hypotheses for future tests.
- Document and decide: implement, reject or repeat. Write down what you learned.
Common mistakes that lead to false conclusions
- Stopping early. You look every day and stop the test when variant B “wins”. In the first days differences fluctuate a lot; stopping at a chance peak produces false winners.
- Too many variants at once. The more variants or metrics you compare, the greater the chance that one looks good by luck alone.
- Changing several things for no reason. If you test a completely different page, you learn that it won, but not why.
- Slicing after the test. “Among women in Cluj, variant B won” is often just noise if the segment was not planned.
- The novelty effect. A new variant can attract attention at first, and the effect fades; this is why duration matters.
- Uneven or faulty traffic split. If one variant receives far fewer visitors than it should, it points to a technical randomisation problem.
- Ignoring guardrail metrics. A variant can raise form submissions while lowering lead quality.
- No documentation. A year later nobody remembers what was tested, and the same tests get repeated.
Key terms
Control: the current version, which you compare against. Null hypothesis: the assumption that there is no real difference between the variants. Statistical power: the chance of detecting a real difference if one exists. Minimum detectable effect: the smallest improvement the test is designed to observe. Confidence interval: the range in which the true effect probably lies. Randomisation: assigning visitors to variants at random.
How we approach it
Before proposing a test, we check that measurement is correct and that traffic allows conclusions. If it does not, we say so openly and suggest fixes based on qualitative research, rather than a test that would only produce noise. When we do test, we document the hypothesis, the calculation and the decision, so the lessons stay with you even when the team changes.
Testing is one component of conversion rate optimisation, alongside analysis, research and implementation. The same discipline applies to advertising; see how we organise creative tests in Facebook Ads.
In short
- An A/B test compares variants in parallel, with visitors split at random.
- Write the hypothesis, the metric and the sample size before launch.
- With low traffic, test bigger changes or use qualitative research.
- Run over whole weeks and do not stop the test when it suits you.
- Statistical significance is a decision threshold, not a guarantee.
- Document every test, including the ones that lost.
How much traffic do I need for an A/B test?
It depends on your conversion rate and on the difference you want to detect. As a rough guide, at a 3% rate and a 20% relative uplift you need tens of thousands of visitors in total. With less traffic, look for bigger changes or use qualitative research methods.
How long should a test run?
Until the calculated sample is reached and, as a rule, for at least two full weeks, so that you cover your audience’s behaviour cycle. Do not stop just because a variant looks like the winner after a few days; early results are unstable and misleading.
Can I test several things at the same time?
Yes, but carefully. If you change several elements in one variant, you learn whether the whole works, not which element matters. Multivariate tests need far more traffic. To begin with, stick to one clear hypothesis per test.
What if the test shows no clear difference?
That is a valid result: it means the change was not large enough to be detected. Record the conclusion, check whether the hypothesis was sound and move on to an idea with bigger impact. A test without a winner still shows you what is not worth the effort.
Which tools can I use for A/B tests?
There are dedicated experimentation platforms, WordPress plugins and testing features in some ad and email platforms. Google Optimize was shut down in 2023, so check which alternatives are available now. Choose a tool that splits traffic properly and integrates with your measurement.
Does A/B testing work for ads and emails too?
Yes, the principles are the same: a hypothesis, one main variable, a sufficient sample and a duration fixed in advance. Ad and email platforms have their own testing tools. Watch out for overlapping audiences and for budgets too small to generate enough events for conclusions.