Why significance testing matters
Conversion rates bounce around naturally. Run the same page as A and B and one will "win" by chance. Without a significance test, teams regularly declare winners off a handful of extra conversions, roll out changes that do nothing, and wonder why the promised uplift never appears.
How to run a trustworthy test
- Decide your sample size in advance based on your baseline rate and the smallest uplift worth detecting.
- Run for full weeks. Behaviour differs between weekdays and weekends, so cover at least one and ideally two full cycles.
- Do not stop the moment you hit 95%. Repeatedly checking and stopping early inflates false positives.
- Test one meaningful change at a time, so you know what caused the result.
- Check segments such as mobile and desktop before rolling out, in case a winner overall loses for a key audience.
What to test first
Focus on changes big enough to matter: the headline and offer, the call to action, form length, social proof and page layout above the fold. For paid search, testing landing pages alongside ad copy often delivers the biggest gains, because improving conversion rate lowers your effective cost per lead.