P-value Calculator
Compare two groups, such as a control and a variant, and find out how likely a difference this large would be if there were no real effect. Enter proportions or means, pick a Z-test or a T-test, and read off the p-value and significance level.
Related calculators
Created by the Reeliy Editorial Team. Last updated: Oct 19, 2026
How to use the p-value calculator
The calculator tests whether the difference between two independent groups, called A (the baseline or control) and B (the treatment or variant), is larger than chance alone would comfortably produce. It covers the two most common situations in experiments: a difference in proportions (conversion rates, event rates, defect rates) and a difference in means (revenue, time, weight, scores).
- Choose Difference in proportions or Difference in means under P-value for.
- Choose whether you want inference about the absolute difference (B − A) or the relative difference ((B − A) ÷ A, the percent change).
- Choose a Z-test or a T-test.
- Enter the data and press Calculate Statistic.
Entering proportions
Give the sample size of each group and the outcome in whichever form you have it: as proportions (0.05), as percentages (5%), or as a count of events (50 out of the sample). All three are converted to the same proportions internally, so choosing the matching format only saves you arithmetic. Proportions must lie between 0 and 1.
Entering means
Either paste the raw observations for each group, separated by commas, spaces, tabs or new lines (a column copied from a spreadsheet works), or enter summary statistics: sample size, mean and standard deviation. At least two observations per group are needed, and a group whose values are all identical (standard deviation of zero) cannot be tested. With raw data the output also lists the sample sizes and the means.
Z-test or T-test
Use the Z-test when the test statistic can be treated as normally distributed, which the central limit theorem usually justifies from about thirty observations per group unless the data are extremely skewed. Use the T-test for small samples, where the extra uncertainty from estimating the spread matters.
For means the calculator also decides how to estimate the standard error. It pools the two groups' variances (the classic Student approach) when the group sizes are within about 5% of each other, and it switches to the unequal-variance Welch approach when they differ by more than 5%, or when the sizes differ and either group has 30 or fewer observations. The row label tells you which was used: Pooled SEM or SEM, and the degrees of freedom are fractional for Welch.
What the output means
- P-value (one-sided). The probability of a difference at least as large as the one observed, in the direction observed, if the null hypothesis is true. The label shows which null was tested: B ≤ A when B came out higher, B ≥ A when it came out lower.
- Two-sided P-value. Shown for the Z-test only. It counts extreme differences in both directions and is twice the one-sided value.
- Z-score or T-score, and for the T-test the degrees of freedom.
- Significance level. The one-sided confidence in a real difference, 100% × (1 − p) when B is above A.
- SEM (standard error) for means, and the observed difference (B − A), or the relative difference for percent-change inference.
What is a p-value?
A p-value is the probability, computed on the assumption that the null hypothesis is exactly true, of obtaining a result at least as extreme as the one you actually saw. The null hypothesis is the model you are prepared to drop only if the data are hard to reconcile with it, typically that the treatment has no effect or a harmful one.
A small p-value means the data would be unusual under the null. It does not give the probability that the null is true, the probability that your result is a fluke, or the size or importance of the effect. Those are common misreadings; the American Statistical Association has published a statement on them (see the references).
To turn a p-value into a decision you compare it with a threshold fixed in advance, the significance level α, commonly 0.05. A result with p < α is called statistically significant at that level. A p-value of 0.05 corresponds to a Z-score of about 1.645 in a one-sided test; the p-value to Z-score calculator converts between the two.
The formulas
Every test here reduces the data to a standardised distance between the groups, a Z or T score, and then reads the p-value from the tail of the matching distribution.
Two proportions. With p₁ and p₂ the observed proportions and p̂ the pooled proportion across both groups:
Z = (p₂ − p₁) ÷ √( p̂ (1 − p̂) (1 ÷ n₁ + 1 ÷ n₂) )p̂ = (p₁n₁ + p₂n₂) ÷ (n₁ + n₂).
Two means. With means m₁ and m₂, standard deviations s₁ and s₂, and sample sizes n₁ and n₂, the unequal-variance standard error is:
SE = √( s₁² ÷ n₁ + s₂² ÷ n₂ ), T = (m₂ − m₁) ÷ SEThe pooled version replaces both variances by one pooled variance.
The one-sided p-value is the area of the tail beyond the observed score: the upper tail 1 − Φ(Z) of the standard normal for a Z-test, or of a t distribution with the relevant degrees of freedom for a T-test. The Welch degrees of freedom are (s₁²/n₁ + s₂²/n₂)² ÷ ( (s₁²/n₁)²/(n₁ − 1) + (s₂²/n₂)²/(n₂ − 1) ). The two-sided p-value doubles the one-sided one.
Worked examples
An A/B test on conversion. Each group has 1,000 visitors. The control converts at 10% and the variant at 12%. The pooled proportion is 0.11, so the standard error is √(0.11 × 0.89 × 2 ÷ 1000) = 0.013993. The difference of 0.02 is Z = 0.02 ÷ 0.013993 = 1.4293 standard errors. The one-sided p-value is 0.0765 and the two-sided one is 0.1529, so the significance level is 92.35%. A two-point lift looks promising, but at this sample size it does not clear a 5% threshold.
Comparing means. Group A has 40 observations with mean 10.5 and standard deviation 3; group B has 50 observations with mean 12 and standard deviation 4. The sizes differ by 25%, so the unequal-variance standard error applies: √(9 ÷ 40 + 16 ÷ 50) = √0.545 = 0.7382. The difference is 1.5, so Z = 1.5 ÷ 0.7382 = 2.0319, giving a one-sided p-value of 0.0211, a two-sided p-value of 0.0422 and a significance level of 97.89%. This clears the usual 5% bar in either form of the test.
Relative difference (percent change)
Many experiments are judged by percent lift rather than by the raw gap. The relative difference is (B − A) ÷ A, and A is itself an estimate with sampling error, so dividing by it adds uncertainty. For the same data, inference about the relative difference therefore gives a larger standard error and a higher p-value than inference about the absolute difference.
Use the setting that matches the claim you will make: test the relative difference to report "conversion rose by 20%", and the absolute difference to report "rose by 2 percentage points". The relative-difference output shows the percent change as a proportion, for example 0.20 for +20%.
Conditions for a valid result
- Fixed sample size. Decide the sample size or the stopping time before you look at the data. Peeking and stopping when p dips below 0.05 inflates the real false-positive rate far above the nominal one.
- One comparison. The calculator tests a single pair of groups on a single outcome. With several variants or several metrics, adjust for multiple comparisons.
- Independent observations. Each observation should belong to one group only and not influence the others. Repeated measures on the same units need a different test.
- Adequate planning. A non-significant result from a small sample is weak evidence of no effect. Plan the size of the test with the power and sample size calculator.
Read a p-value alongside an estimate of the effect and its uncertainty. The confidence interval calculator gives the range of effects compatible with your data, which says how big the difference is, not just whether it is nonzero.
Frequently asked questions
What p-value counts as statistically significant?
There is no universal cut-off. The threshold, called α, must be chosen before the test, and 0.05 is the most common convention. Stricter fields use 0.01 or lower, and exploratory work sometimes accepts 0.10.
Should I use the one-sided or the two-sided p-value?
Use the one-sided value when you only care about whether B is better than A and have committed to that direction in advance. Use the two-sided value when a difference in either direction would matter. The two-sided p-value is twice the one-sided one.
When should I choose the T-test instead of the Z-test?
Choose the T-test for small samples, roughly fewer than 30 observations per group, where estimating the standard deviation adds noticeable uncertainty. With larger samples the two give nearly the same answer.
What does a significance level of 95% mean here?
It is the complement of the one-sided p-value, 1 − p, when B is above A. A level of 95% corresponds to p = 0.05. It is not the probability that the effect is real.
Why is the p-value for relative difference larger?
Percent change divides the difference by the control's value, which is itself uncertain. That extra uncertainty widens the standard error, so the same data are less convincing when judged on relative lift.
References
- Wasserstein, R. L., Lazar, N. A. (2016). The ASA statement on p-values: context, process, and purpose. The American Statistician 70(2), 129–133.
- Welch, B. L. (1947). The generalization of "Student's" problem when several different population variances are involved. Biometrika 34, 28–35.
- Fisher, R. A. (1935). The Design of Experiments. Oliver and Boyd.
Cite this calculator and page
If you use this page in your work, you can cite it as:
Reeliy Editorial Team. (2026). P-value Calculator. Reeliy. https://reeliy.com/calculators/p-value-significance-calculator“P-value Calculator.” Reeliy, 2026, https://reeliy.com/calculators/p-value-significance-calculator.