Bayesian A/B Testing Calculator

Use this free bayesian A/B testing calculator to find out if your test results are statistically significant. For each variation you tested, input the total sample size, and the number of conversions.
Samples
The number of users, sessions or impressions depending on your KPI
Conversions
The number of clicks or goal completions
A
B
CALCULATE
Add Variation
You've reached the maximum of 10 variations.
Winner significance level
Variations that exceed this threshold are declared the winner of the test
Samples must be greater or equal to Conversions
Variation A is the winner!
Based on 95% significance level
Probability to be Best
Each variation's long-term probability to out-perform all other live variations, given collected data since the creation or change of any variation included in the test
96.64%
3.36%
0%
10%
20%
30%
40%
50%
60%
70%
80%
90%
100%
Samples
Conversion
Conversion Rate
Probability to be Best
Each variation's long-term probability to out-perform all other live variations, given collected data since the creation or change of any variation included in the test
Expected Loss
Assuming I declare the variation as a winner, and I am wrong, how much am I expected to lose in the long term, in term of % vs the variation which is actually the best
A
8,500
1,500
17.647%
96.64%
0.89%
B
8,500
1,410
16.588%
3.36%
6.20%
Posterior simulation of difference
The distribution of conversion rates given the sample size collected so far
120%
110%
100%
90%
80%
70%
60%
50%
40%
30%
20%
10%
0%
0%
10%
20%
30%
40%
50%
60%
70%
80%
90%
100%
The Power of Bayesian A/B Testing Bayesian A/B Test Duration & Sample Size Calculator Frequentism and Bayesianism: A Practical Introduction The Importance of Statistical Significance in A/B Tests How Not To Run An A/B Test Definition of Probability to Be Best in A/B Testing