How many variants to test
The number of variants you can meaningfully test depends on your audience size and primary metric. Smaller audiences need larger differences between variants to reach a conclusion.
Flow and automation campaigns (baseline 1-5%)
| Your baseline conversion rate | Minimum difference to detect | Audience needed per variant |
|---|---|---|
| 1% | 0.5 percentage points | ~6,300 |
| 2% | 1.0 percentage point | ~3,100 |
| 3% | 1.0 percentage point | ~4,700 |
| 5% | 2.0 percentage points | ~1,900 |
Broadcast and campaign sends (baseline 0.05-0.2%)
Conversion rates for one-off campaign sends are typically much lower than flows. At these rates, liftstack automatically switches to relative effect sizes rather than fixed percentage point targets:
| Your baseline conversion rate | Relative lift to detect | Audience needed per variant |
|---|---|---|
| 0.05% | 100% (doubling) | ~32,000 |
| 0.10% | 100% (doubling) | ~16,000 |
| 0.20% | 50% | ~12,800 |
| 0.50% | 50% | ~5,100 |
Automatic sample size guidance
When you set up a campaign, liftstack automatically shows a sample size guidance card after your audience is synced. This tells you whether your audience is large enough for the number of variants you are testing, using your workspace’s historical conversion rate (or a 3% default if you have no history).
For low conversion rate campaigns (below 0.5%), the guidance automatically uses relative effect sizes and shows an additional warning with advice on alternative metrics or campaign types.
Why fewer can be better
Each additional variant dilutes the traffic per variant, requiring more exposures for a definitive verdict. Two well-designed variants that test a clear strategic question will produce more actionable insights than five variants with minor copy differences.