Power & Sample Size Calculator

How to use the power and sample size calculator

The calculator answers one of two planning questions. In Sample Size Calculation mode you say how much power you want and it returns the number of observations needed. In Power Analysis mode you give a sample size and it returns the probability that the test would detect your effect of interest. Both assume the test will be a one-sided Z-test, except for equivalence designs, which are inherently two-sided.

1. Pick the mode: sample size or power.
2. Say what the experiment compares: a single sample against a reference value, or the difference between samples (a control group against one or more treatment groups).
3. Choose the kind of data: means (continuous) or proportions (binomial).
4. Choose the alternative hypothesis (superiority, non-inferiority or equivalence) and whether you will draw inference about the absolute or the relative difference.
5. Fill in the parameters below and press Calculate.

Why sample size and power matter

Statistical power is the probability that a test rejects the null hypothesis when a particular alternative is true: 1 − β, where β is the type II error rate, the chance of missing a real effect. Larger samples, bigger effects, cleaner data and a more lenient α all raise it.

Doing the arithmetic before collecting data protects you from two expensive outcomes: a test too small to find anything, and a test far larger than it needs to be. It also forces you to state, in advance, the smallest effect you actually care about. Power computed afterwards from the observed result adds nothing beyond the p-value, so treat it as a planning tool only.

Sample size formulas

Let z₁₋α and z₁₋β be the standard normal quantiles for the chosen error rates, σ the standard deviation and δ the effect the test must resolve. For a one-sample test of a mean:

n = ( (z₁₋α + z₁₋β) × σ ÷ δ )²

For two groups of equal size, the difference of two means has twice the variance, so each group needs:

n per group = 2 × ( (z₁₋α + z₁₋β) × σ ÷ δ )²

For two proportions the variance depends on the proportions themselves. With baseline p₀ and expected treatment proportion p₁ = p₀ + margin + MDE for a superiority test (for non-inferiority the margin is subtracted), and δ the MDE:

n per group = (z₁₋α + z₁₋β)² × ( p₀(1 − p₀) + p₁(1 − p₁) ) ÷ δ²

For equivalence the test must show that the true difference lies inside ±margin, which is done with two one-sided tests. The calculator then uses z₁₋α⁄₂ in place of z₁₋α and replaces δ with margin − |MDE|, the room left between the expected effect and the edge of the equivalence zone. Power is the inverse calculation: Φ(δ ÷ SE − z₁₋α), where SE is the standard error of the difference at the planned sample size.

Worked examples

One sample, means. The standard deviation is 3 and you want to detect a shift of 1 with 80% power at α = 5%. Then n = (2.4865 × 3 ÷ 1)² = 55.6, which rounds up to 56 observations.

Two groups, means. The same assumptions for a control and a treatment group need 2 × 55.6 = 111.3 per group, so 112 per group and 224 in total. Raising power to 90% lifts the figure to 155 per group, and 95% to 195.

Two proportions. Baseline conversion is 10% and you want to detect a rise to 12% (an MDE of 0.02). The variance term is 0.10 × 0.90 + 0.12 × 0.88 = 0.1956, so n = 6.18 × 0.1956 ÷ 0.02² ≈ 3,023, rounded up to 3,024 per group, 6,048 in total. Detecting the same two-point lift with 90% power needs about 4,188 per group.

Equivalence. With σ = 3, an expected difference of 1 and an equivalence margin of 2, the room to the edge is 2 − 1 = 1. The two-group size is 2 × 9 × (1.96 + 0.8416)² ÷ 1² = 141.3, so 142 per group.

Power for a given size. With 112 observations in each group (224 in total) and the settings above, the power is 80.22%, matching the sample size found earlier.

Superiority, non-inferiority and equivalence

• Superiority asks whether the treatment beats the baseline (optionally by more than a stated margin). It is a one-sided test.
• Non-inferiority asks whether the treatment is not worse than the baseline by more than the margin. It is chosen when the new option has other advantages, such as lower cost or fewer side effects, and is also one-sided.
• Equivalence asks whether the difference is small enough to count as none, within plus or minus the margin. It is evaluated with two one-sided tests, so the calculator uses the two-sided critical value.

The MDE is measured from the boundary of the null. With a superiority margin of 1 over a baseline of 10, and a treatment you hope will reach 13, enter an MDE of 2, because margin plus MDE gives the full distance of 3.

Absolute versus relative difference

An absolute difference is B − A. A relative difference is (B − A) ÷ A, the percent change. The baseline A is itself an estimate with error, so dividing by it adds variance: a test about the relative effect needs somewhat more data than one about the equivalent absolute effect. The calculator adds an allowance for this when you choose relative inference. For means with relative inference, enter the MDE and margin as proportions or percentages of the baseline mean.

Conventions in this calculator

• Power mode and the total sample size. The size you enter is a total that is divided between group A and group B according to the sampling ratio, even when the design is set to a single sample. A single-sample power figure therefore describes the matching two-group layout and comes out lower than a textbook one-sample calculation would give. For single-sample planning, rely on sample size mode.
• Several treatment groups. An approximate Dunnett-style adjustment is applied and the results are per-group and total sizes. Power analysis covers a single comparison only. Check multi-group figures in dedicated software.
• Sampling ratio. Group A is kept at the size computed for equal groups, and group B is group A divided by the ratio; a ratio of 2 turns 112 and 112 into 112 and 56 in the two-means example.

Sample size and power figures are planning estimates built on assumptions about the baseline, the spread and the effect. They are not a substitute for review by a qualified statistician, particularly for clinical, regulatory or safety-related studies. See the disclaimer.

Once the data are in, test the difference with the p-value calculator and describe its uncertainty with the confidence interval calculator. A pilot sample and the standard deviation calculator can supply the σ you need.