The Test Graveyard

Hypothesis

Rounding up — 'trusted by over 50,000 customers' — will outperform the oddly specific '48,217 orders shipped', because bigger sounds better.

Setup

Headline social proof line on the homepage hero, two versions, 3 weeks to sample.

Result

The vague version lost: −3.4% on add-to-basket from homepage sessions.

What it taught us: Specificity signals honesty; round numbers signal marketing. Visitors discount claims that sound like claims. We now default to precise, verifiable figures everywhere and let the oddness do the persuading.

Hypothesis

Students are price-led, so surfacing weekly rent as the dominant element on listing cards will increase clicks through to room detail pages.

Setup

Listing card redesign: price set in display type above the room photo, distance-to-campus demoted to metadata. Ran 4 weeks to pre-registered sample.

Result

Click-through to room pages −6.2% (significant). Enquiries −4.8%.

What it taught us: Price matters at comparison stage, not discovery stage. Early-funnel students filter on location and photos first; leading with price read as 'budget' and suppressed clicks on mid-market rooms. Price now appears prominently one step later, at room detail level.

Hypothesis

Adding recognised payment and security badges beside the pay button will reduce checkout abandonment by reassuring hesitant buyers.

Setup

Row of five badges (card schemes, SSL, money-back) directly under the pay CTA. Ran to full sample over 3 weeks.

Result

Completed checkouts +0.4% (not significant). No movement in any guardrail.

What it taught us: For an established brand with a normal-looking checkout, trust was not the binding constraint — nobody doubted the site was legitimate. Badges are a fix for perceived-risk problems, which we hadn't demonstrated existed. Should have run the exit survey first.

Hypothesis

A visible countdown to the offer deadline will create urgency and lift enquiry conversion.

Setup

Live countdown timer added to promotional landing pages. Won +9% at 95% in week two; we kept a 90/10 holdout running post-rollout.

Result

+9.1% at test end; +1.2% against holdout by week eight.

What it taught us: Classic novelty effect — returning visitors responded to the new moving element, then habituated. The residual uplift didn't justify the maintenance cost or the slightly pushy tone. Retired. Long-running holdouts on 'urgency' wins are now standard practice.

Hypothesis

Fewer form fields means less friction, so a 4-field enquiry form will produce more enquiries than the 9-field version.

Setup

Removed move-in date, budget, room-type preference, phone and 'how did you hear'. Ran 5 weeks to full sample, with enquiry-to-booking tracked as a guardrail.

Result

Enquiries +11% — but bookings per enquiry −23%, and net bookings −8%.

What it taught us: The 'friction' was doing a job: qualifying intent and giving the lettings team what they needed to respond well. Optimising the form submit was optimising the wrong metric. Primary metrics should sit as close to revenue as traffic allows.

Hypothesis

An AI assistant answering pricing questions in-page will lift demo bookings by resolving objections at the moment they occur.

Setup

Chat widget bottom-right on pricing and comparison pages, seeded with pricing FAQ content. 6 weeks to sample.

Result

Demo bookings +0.9% (not significant). 3.1% of visitors opened the widget; transcripts showed most questions were already answered on the page.

What it taught us: The widget answered questions the page already answered — the real finding was in the transcripts: the two questions the page didn't answer (contract length, migration support). Adding those to the page is the follow-up test. Sometimes a test's value is the qualitative exhaust, not the metric.