Digital Marketing

What Is A/B Testing?

Definition

A method of comparing two versions of a web page, email, or ad by showing each version to a different group of users to determine which performs better.

Why It Matters

A/B testing removes guesswork from marketing decisions. Instead of debating which headline is better in a team meeting, you let real user behaviour decide — based on statistically significant data rather than opinion or assumption.

How It Works

A/B testing splits traffic evenly between a control version (A) and a variant (B), with only one element changed between them. The winning version is determined by a primary metric (conversion rate, click-through rate, revenue per visitor) once statistical significance is reached (typically 95% confidence). Tools include Google Optimize, VWO, Optimizely, and built-in testing in email platforms.

Real-World Example

An online retailer A/B tests two product page headlines: "Buy the Comfort Series Mattress" (A) vs "Wake Up Pain-Free — The Comfort Series Mattress" (B). After 14 days and 8,000 visitors, version B achieves 34% higher add-to-cart rate with 99% statistical confidence — and becomes the permanent version.

Quick Facts

Test only one variable at a time — changing multiple elements makes it impossible to identify what drove the result
Most A/B tests require at least 1,000 visitors per variant to reach statistical significance
Winning variants should be treated as the new control and tested further — optimisation is never finished
A/B testing email subject lines is the highest ROI and lowest risk test available to most marketing teams