06.4.3 · Skill in Experiment design
Randomization
Assigning users to groups by chance so the groups are comparable.
Random assignment balances known and unknown differences between groups. It has to be consistent, so a user stays in one group, and done at the right unit, such as user, team or account. Broken assignment invalidates the whole experiment.
Resources
2- docsStart hereHow randomization works in Amplitude Experiment ↗Amplitude docs · 2026 · 3 min
A compact look at hashing-based assignment and the two-step decision of who enters an experiment and which variant they get.
- docsGo deeperHow evaluation works ↗Statsig docs · 6 min
How users are assigned to groups with a salted hash, so the same user always gets the same variant without stored state.
Tools
2- ToolOpen sourceGrowthBook
Open-source feature flags and A/B testing that analyzes results from your own data warehouse.
- ToolFreemiumStatsig
Feature flags, experiments and product analytics with built-in statistics such as variance reduction.