A/B testing & CRO consultancy · UK
Most A/B tests lose.
Ours tell the truth.
We design, run and read experiments for e‑commerce and property brands — with proper sample sizes, no peeking, and no dashboard theatre. When a test loses, we say so. That's the point.
You were randomly bucketed into a variant of this site. Toggle it in the header — the headline changes, the honesty doesn't.
What we do
01 · Programme design
Testing programmes, not one-off tests
Hypothesis backlogs, prioritisation, sample size planning and a cadence your traffic can actually support. Built to compound, not to fill a slide.
02 · Build & run
Experiments built and QA'd properly
Clean implementations, guardrail metrics, A/A validation and no flicker. We run on our own platform, ABexis, or in your existing tooling.
03 · Honest analysis
Results you can defend in a board meeting
Pre-registered success criteria, no peeking, no post-hoc segment fishing. Losers documented as carefully as winners — see the Graveyard.
From the blog
All articles →
Most A/B tests lose. That's not a bug — it's the entire point
Industry win rates hover around one in four or five. If your programme wins more than that, something is probably wrong with your programme, not right with it.
Peeking: how checking your test every morning quietly ruins it
A test checked daily and stopped at the first green light isn't running at 95% confidence. It's often running closer to a coin flip — and the dashboard will never tell you.
The sample size maths nobody does before launching a test
Five minutes with a power calculation kills more bad tests than any amount of post-hoc statistical sophistication. Here's the back-of-envelope version.
The novelty effect: why your winning test faded after four weeks
A variant that wins in week one and evaporates by week six probably didn't stop working. It never worked — it was just new.
Copy testing beats colour testing: an argument from effect sizes
The famous button-colour test is CRO's origin myth, and it has taught a generation of marketers to test the wrong things. Words carry meaning. Hue mostly doesn't.
The Texas sharpshooter problem, or: why your test 'won with iOS users in Leeds'
Slice any flat test into enough segments and one of them will always be a winner. Post-hoc segmentation is where inconclusive tests go to be laundered into success stories.
Things nobody else publishes
Public archive
The Test Graveyard 🪦
Every consultancy shows you their winners. We publish our losers — real hypotheses that died, and what each one taught us.
Interactive tool
The Peeking Simulator
Check your test every day and you'll "win" tests that aren't winning. Run 1,000 simulated A/A tests and watch the false positives pile up.
This website
The site is a split test
You were bucketed into variant A or B on arrival. The header toggle lets you switch. It's a small joke with a serious point: variants should differ by one thing.