Type I error

Type I error means rejecting a true null hypothesis. The null hypothesis is a formal claim such as “there is no difference” or “the treatment has no effect”; when that claim is true in the population, a rejection is a false positive.¹ The significance level α is the maximum long-run error probability specified before testing, assuming the model and null hypothesis are correct.

An α of 0.05 does not mean that a reported result has a 5% probability of being false. It means that, over repeated samples and tests conducted under the null, the procedure would reject about 5% of the time if its assumptions hold. The actual probability that a hypothesis is true requires different information, such as a prior probability and a statistical model; the p-value is not that probability.²

A Type I error occurs when a p-value falls below the prespecified significance threshold even though the null hypothesis is true. A p-value measures how incompatible the observed data, or more extreme data, are with the null model; it does not measure the size or practical importance of an effect.²

The error rate belongs to a testing procedure, not merely to an isolated number. Its validity depends on features such as random sampling, independence, the chosen probability model, and whether the analysis was selected before examining the data. Repeatedly trying alternative outcomes, subgroups, transformations, or statistical models can make at least one small p-value likely, even when every null hypothesis is true. This practice is often described as multiple testing or researcher degrees of freedom.³

Researchers control Type I error by defining the decision rule in advance and adjusting it when many hypotheses are tested. A single test at α = 0.05 targets a 5% error rate under its assumptions, whereas testing many independent null hypotheses at that threshold raises the probability of one or more false rejections.

Common remedies include Bonferroni adjustment, which sets a per-test threshold such as α divided by the number of tests, and procedures controlling the family-wise error rate or the false discovery rate. These methods make different trade-offs: family-wise control is stricter, while false-discovery-rate procedures allow a controlled proportion of false discoveries among rejected hypotheses. Prespecification, transparent reporting, replication, confidence intervals, and correction for multiplicity complement—not replace—these procedures.

Type I error is distinct from Type II error, which is failing to reject a false null hypothesis. Tightening α generally reduces false positives but can reduce statistical power and increase false negatives unless the sample size grows. Thus, the choice of threshold is a design decision involving consequences, measurement quality, and the relative costs of the two errors rather than a purely mathematical convention.

The distinction is especially consequential in clinical trials, genomics, astronomy, and other fields conducting large numbers of tests. A statistically significant result can still have a negligible effect size, poor external validity, or a clinically unimportant consequence. Conversely, a nonsignificant result does not prove that no effect exists. The American Statistical Association therefore recommends interpreting p-values in context, reporting effect sizes and uncertainty, and avoiding a simple significant-versus-nonsignificant dichotomy.²

Null hypothesis

A formal statistical claim treated as the reference condition, often that an effect or difference is absent.

p-value

The probability, assuming the null model, of observing data at least as extreme as those obtained.

Statistical power

The probability of rejecting a false null hypothesis under a specified alternative.

Family-wise error rate

The probability of making one or more Type I errors across a family of tests.

False discovery rate

The expected proportion of false rejections among all rejected hypotheses.