Copy testing beats colour testing: an argument from effect sizes

Every intro-to-CRO talk includes the button colour story — the green versus red test, the 21% uplift, the millions attributed. It's a fine story. It's also fifteen-plus years old, from an era when most sites were genuinely broken, and it has left the industry with a lasting bias towards testing appearance when the leverage is overwhelmingly in language.

Why colour tests usually produce nothing

For a change to alter behaviour, it has to change something in the visitor's head: their understanding, their confidence, their sense of what happens next. Swapping a button from teal to orange changes almost none of that — unless the original failed a basic visibility or contrast check, in which case you've found a defect, not an insight.

That's why mature testing programmes report colour and minor styling tests clustering around zero effect. And near-zero true effects have a nasty statistical property: the sample size needed to detect them is enormous, so these tests either run forever or get called early on noise. The colour test isn't just low-value; it's the exact category of test most likely to produce a false winner.

Why copy moves numbers

Copy is where the visitor's actual questions live. What does this button do? What am I committing to? What does it cost? What happens after I click?

Consider "Submit" versus "See rooms & prices" on a student accommodation enquiry form. The first is a system word — it describes what the form does. The second answers the visitor's question — it describes what they get. Changes like that routinely move numbers by high single digits or better, because they change the perceived transaction, not the pixels.

The highest-leverage copy tests we see, repeatedly:

• Verb-first, outcome-focused CTAs ("Get my valuation", "Check availability") over system labels ("Submit", "Continue")
• Anxiety-reducers at the point of commitment — "No deposit taken today", "Cancel anytime before check-in" — placed next to the button, not buried in an FAQ
• Headlines that state the offer plainly versus headlines that are clever. Clarity beats cleverness so reliably it's barely worth testing any more, and yet
• Specificity — "Reply within 2 working hours" beats "Fast response"; "12,400 students housed since 2015" beats "trusted by thousands"

The deeper principle: test meaning, not decoration

The button colour myth persists because colour tests are easy — no copywriter, no legal sign-off, no argument about tone of voice. But test difficulty and test value are correlated. Changes that require someone to think hard about what the customer needs to hear are exactly the changes big enough to detect and worth detecting.

A useful filter for any proposed test: if this variant wins, what will we have learned about our customers? "They prefer orange" is not a learning you can build on. "They enquire more when we surface total cost up front" changes your pricing pages, your ads, your sales scripts.

When appearance is worth testing

To be fair to pixels: layout, imagery and visual hierarchy can carry meaning — a photo of the actual room versus a stock lifestyle shot, a price shown before versus after the fold. Those are worth testing precisely because they change what the visitor understands. That's the line: appearance in service of meaning, yes. Hue for hue's sake — save your traffic.