Experiment Velocity
Experiment velocity measures how many tests you ship per period — and it's the single biggest predictor of CRO program ROI. Here's the formula, benchmarks, and how to raise it.
Experiment Velocity
The number of A/B or multivariate tests an organization ships per period — usually counted per month.
Experiment velocity is how many controlled experiments your team starts (or concludes) in a given window, almost always expressed as tests per month. It's a throughput metric, not an outcome metric, but the two are tightly linked: with a fixed win rate and average lift, more tests mechanically produce more annual revenue.
It sits at the centre of any serious experimentation strategy because it's the one input you mostly control. Win rate and effect size depend on hypothesis quality and traffic; velocity depends on process, tooling, and dev dependencies — the things a CRO program can actually fix.
Velocity matters because wins compound. If you ship 4 tests a month, win 25% of them, and the average winner adds 5% lift to the affected funnel step, you're stacking roughly 12 winning treatments a year onto your site. Each one persists, so the gains accumulate against a steadily improving baseline.
The trap is treating velocity as a vanity number. Shipping 12 underpowered tests a month with no traffic to call them produces zero learning. The honest version of the metric is concluded, properly-powered experiments per month — tests that reached their pre-registered sample size and gave you a decision.
Annual Lift ≈ Velocity × 12 × Win Rate × Avg Lift
Velocity
Experiment velocity
Concluded experiments per month
Win Rate
Win rate
Share of tests that produce a statistically significant positive winner
Avg Lift
Average lift per winner
Mean conversion-rate increase of winning treatments on the affected step
A Shopify apparel store running checkout and PDP tests through a small CRO pod.
Velocity (tests/month): 4
Win rate: 25%
Average lift per winner: 5%
→ ≈ 15% compounded annual lift on the tested funnel steps
Four modest tests a month at industry-average win rates produces double-digit annual conversion gains. Doubling velocity to 8/month roughly doubles the annualised lift — which is why velocity is usually the highest-leverage input to optimise.
How fast is fast? It depends on traffic and program maturity. A store doing 30k sessions a month can't run 10 concurrent tests — there isn't enough power. The benchmarks below are realistic targets for stores in the €1M–€15M revenue band on Shopify, WooCommerce, or Magento.
Experiment velocity benchmarks by program maturity
| Program stage | Tests / month | Typical win rate | Avg lift per winner |
|---|---|---|---|
| Just starting (0–6 months) | 1–2 | 15–20% | 3–4% |
| Established (6–18 months) | 3–5 | 20–25% | 4–6% |
| Mature (18+ months) | 6–10 | 25–30% | 4–7% |
| Best-in-class | 10–15+ | 30–35% | 5–8% |
Most stores stall in the established band because every new test requires a developer ticket. Removing that dependency — usually with a visual editor and a lightweight snippet — is what unlocks the jump from 3/month to 8/month. The win rate tends to climb too, because faster iteration means more hypotheses tested against the same drop-off.
Experiment velocity FAQ
Count the number of experiments that concluded (reached their pre-registered sample size or stopping rule) in a calendar month, then average across the last 3–6 months to smooth out noise. Tests that were aborted or never powered shouldn't count.
For stores doing €1M–€15M in annual revenue, 3–5 concluded tests per month is the established benchmark, and 6–10/month puts you in the top quartile. Below 1/month usually means you don't have a program yet — you have occasional tests.
Yes, in most programs. Win rate is hard to move past ~30% because it's bounded by hypothesis quality and statistical noise. Velocity has much more room to grow — going from 2 to 6 tests a month triples your annual lift even if win rate stays flat.
Remove dev dependencies with a visual editor, build a backlog so you're never idea-starved, run mutually exclusive tests on different pages in parallel, and set a strict maximum test duration (usually 4 weeks) so under-powered tests don't camp out indefinitely.
Only if you run them on overlapping audiences or peek at results early. Parallel tests on different pages or different traffic segments are fine. The real risk is shortening tests below their required sample size to hit a velocity target — that produces noise, not learning.
Velocity is one of three levers — win rate and average lift are the others — that together produce program ROI. An experimentation strategy decides which levers to invest in given your traffic, team size, and current bottleneck. Most early-stage programs should prioritise velocity first.
Roughly 50k–100k sessions/month on the tested pages, assuming a baseline conversion rate around 2–3% and a minimum detectable effect of 10%. Lower traffic forces longer tests, which caps velocity regardless of how fast your team can build them.
Most teams count them as one test each, but flag MVTs separately because they need more traffic and take longer. If you're using MVTs to substitute for sequential A/B tests, your effective velocity is lower than the raw count suggests.
Until it reaches the sample size required for your minimum detectable effect, with a floor of one full business cycle (usually 2 weeks to cover weekday/weekend patterns) and a ceiling of 4 weeks. Tests that need more than 4 weeks usually have an MDE that's too ambitious for your traffic.
Less directly. Below ~30k monthly sessions you're traffic-bound, not idea-bound — running more tests just spreads thin power across more variants. Focus on bigger bets (radical redesigns, pricing, offer tests) and accept a velocity of 1–2/month.
Test ideas before you ship them
Run unlimited A/B tests, attach hypotheses to outcomes, and build a searchable archive of what works — and what doesn't.