Stable3 min read
A/B Testing
Run disciplined experiments.
A/B Testing phase plans controlled experiments — hypotheses, metrics, sample size, and ethics — not random button colour tweaks.
Opinion wars waste time. Disciplined tests settle pricing, onboarding, and messaging debates.
When traffic or deal volume supports tests; after analytics instrumentation.
- Hypothesis tied to OKR or journey metric.
- Primary metric and guardrails.
- Power and duration estimate.
- Run and analyse with pre-registered rules.
- Document result in validation report.
- Hypothesis falsifiable.
- Guardrails protect revenue/retention.
- Peeking early — false winners.
- Testing too many variants at once.
- No ethical review for sensitive tests.
/abtest phase. Links OKR-, CJM-; experiment logs to VALR-.
Sources & further reading
- Kohavi, R., Tang, D., & Xu, Y. (2020). Trustworthy Online Controlled Experiments. Cambridge University Press.