Type I and type II errors

Type I and type II errors arise because hypothesis tests make decisions from samples rather than observing an entire population. A Type I error occurs when a test rejects the null hypothesis even though it is true; its long-run probability is the significance level α. A Type II error occurs when the test does not reject a false null hypothesis; its probability is β for a particular alternative and study design.¹

The terminology concerns decisions, not whether a hypothesis is intrinsically true or false. “Fail to reject” is therefore more precise than “accept,” because a nonsignificant result may reflect limited information rather than evidence that the null hypothesis is correct. The four possible outcomes form a two-by-two table: correct rejection, correct non-rejection, false rejection, and false non-rejection.

Reducing one error probability generally affects the other unless the sample size, effect size, or measurement precision also changes. With the data and test procedure fixed, making α smaller usually makes rejection harder and raises β; increasing sample size can reduce both error probabilities for a specified effect.²

Power depends on α, sample size, the true effect size, outcome variability, and the alternative being considered. A power analysis conducted before data collection can estimate the sample required to detect a scientifically meaningful effect, while a post hoc calculation based only on an observed result is often uninformative. Multiple comparisons, optional stopping, and selective reporting can increase the effective Type I error rate unless planned controls are used.³

A statistically significant result does not mean that the null hypothesis has a fixed probability of being false, and a nonsignificant result does not prove that no effect exists. The p-value is calculated under a specified null model; it is not the probability that a Type I error occurred.³

Practical interpretation should combine the estimate of effect, its confidence interval, the study design, and the consequences of mistakes. In clinical research, a false positive can expose patients to ineffective or harmful treatment, whereas a false negative can delay a beneficial intervention. Regulatory and clinical decisions may therefore use prespecified endpoints, multiplicity adjustments, replication, and thresholds for clinical importance rather than relying on a single p-value.

Type I and type II error rates are not universal properties of a test; they depend on the decision rule, sampling plan, model assumptions, and the particular alternative. β is consequently a function of an effect size, not one number that describes a study in all circumstances. For composite, sequential, or adaptive designs, the error probabilities must be calculated for the complete analysis plan rather than for an isolated final comparison.

Bayesian analysis uses posterior probabilities and credible intervals rather than defining evidence primarily through long-run α and β, although decision-theoretic losses still allow false-positive and false-negative consequences to be represented. In diagnostic testing, the related terms sensitivity and specificity describe performance conditional on disease status; predictive values additionally depend on prevalence, a distinction that prevents direct substitution of diagnostic terminology for hypothesis-test error rates.⁴

Null hypothesis

The proposition tested by a statistical procedure, often representing no effect or no difference.

Alternative hypothesis

A proposition representing an effect, difference, or relationship that contrasts with the null hypothesis.

Significance level

The prespecified maximum long-run probability of rejecting a true null hypothesis, commonly written as α.

Statistical power

The probability of rejecting a false null hypothesis for a specified alternative; it equals 1 − β.

Multiplicity

The inflation of false-positive risk that can occur when several hypotheses, outcomes, or analyses are examined.