What a p-value actually tells you
A p-value is the probability of seeing a test statistic at least as extreme as the one calculated from the data, if the null hypothesis were actually true. That single sentence is the whole definition, and it is worth reading twice, because almost every common misreading comes from dropping the second half of it. A p-value is not the probability that the null hypothesis is true, and it is not the probability that the result happened by chance in some general sense. It is a statement about how surprising the data would be under one specific assumption, nothing more.
This distinction is not a pedantic quibble; it is the exact point the American Statistical Association felt the need to issue a formal statement about in 2016, after decades of the reversed reading creeping into published research and news coverage. Their statement puts it plainly: "P-values do not measure the probability that the studied hypothesis is true." A small p-value says the data would be unusual if the null hypothesis held; it does not, on its own, say how likely any particular hypothesis is to be the right explanation.
Reading the shaded region: one tail or two
The tail choice decides which direction "at least as extreme" points in, and it changes the answer, sometimes by a factor of two. A right-tailed test asks for the probability of a statistic at least this large; a left-tailed test asks for at least this small; a two-tailed test asks for at least this far from the centre in either direction, and is worked out as twice the smaller one-sided probability.
The honest rule is simple: the tail has to be chosen from the hypothesis before the data is seen, not picked afterwards because it produces a smaller p-value. A one-tailed test is defensible when the direction of any real effect is fixed by the question itself, for instance testing whether a new process reduces defects, where an increase was never going to count as a success either way. A two-tailed test is the safer default whenever a difference in either direction would be worth reporting, which covers most everyday comparisons. Switching from two-tailed to one-tailed after glancing at which direction the data leans is one of the more common ways a result ends up looking more convincing on paper than the evidence actually supports.
Choosing z, t or chi-square
The three distributions answer different shapes of question. The standard normal (z) applies when the population standard deviation is genuinely known, or the sample is large enough that the sample standard deviation is a very close stand-in for it; z-tests for a proportion also fall here. The t-distribution, introduced by William Sealy Gosset under the pen name "Student" in 1908, is the right choice whenever the standard deviation has to be estimated from the sample itself, which describes the overwhelming majority of real one-sample and two-sample mean comparisons; its extra spread compared with z, set by the degrees of freedom, accounts honestly for that added uncertainty. The chi-square distribution, developed by Karl Pearson around 1900, turns up for goodness-of-fit tests, tests of independence in contingency tables, and tests about a single variance; because it is built from squared quantities it can never fall below zero and its curve is not symmetric, which is why this page defaults it to a right-tailed test and adds a note whenever two-tailed chi-square is selected.
Degrees of freedom describe how much independent information the statistic was built from. For a one-sample or paired t-test it is usually the sample size minus one; for a chi-square goodness-of-fit test it is usually the number of categories minus one. The z-score calculator is the simpler tool when the question is just "how many standard deviations is this value from the mean", without a formal hypothesis test attached.
Where the 0.05 threshold came from
The 0.05 cut-off did not arrive from any law of probability; it is a convention, and a fairly recent one at that. R.A. Fisher suggested it in his 1925 book Statistical Methods for Research Workers as a convenient round-number threshold, one worth using consistently rather than one derived from first principles. Fisher himself treated it as a useful rule of thumb, not a bright line between "true" and "false", and he expected researchers to weigh the size and context of a result alongside the p-value rather than instead of it.
That history matters because 0.049 and 0.051 describe results that are, in any meaningful sense, almost identical, yet a strict threshold treats one as a discovery and the other as nothing. Alongside 0.05 this calculator always shows the working at 0.01 too, precisely so a result can be read as a matter of degree rather than a single pass-or-fail cut.
How the calculator gets from statistic to probability
None of the three cumulative distribution functions used here has a simple closed algebraic form, so each is computed from a published numerical method rather than read off a table. The standard normal CDF uses the erf-based approximation printed as formula 7.1.26 in Abramowitz and Stegun's Handbook of Mathematical Functions, with a stated maximum error of about 1.5×10⁻⁷, far tighter than the four decimal places shown. The t-distribution's CDF is computed from the regularized incomplete beta function, evaluated by the continued-fraction method set out in Press et al.'s Numerical Recipes; the chi-square CDF comes from the closely related regularized incomplete gamma function, evaluated by a series expansion or a continued fraction depending on the input, exactly as in the same reference. The critical values shown in the reverse lens are not looked up from a separate table at all; they are found by narrowing in on these same curves until the probability matches 0.05 or 0.01, so the forward p-value and the reverse critical value can never quietly disagree with each other.
A calm word on p-hacking
P-hacking describes the practice of trying several plausible ways of analysing the same data, perhaps a different tail, a different subgroup, or a different cut-off for outliers, and reporting whichever version happens to cross the 0.05 line. Each individual choice can look defensible in isolation, but tried enough times, chance alone guarantees that some version of the analysis will turn up a small p-value even when there is no real effect at all. This is not usually deliberate fraud; it is far more often a string of small, reasonable-seeming decisions made after already knowing what the data looks like. The fix is not more suspicion of any single p-value, but deciding the analysis, including the tail, in advance of collecting the data wherever that is practical.
Statistical significance versus practical significance
A tiny p-value only says a difference is unlikely to be pure chance; it says nothing about whether that difference is big enough to matter in practice, and with a large enough sample even a trivially small effect can produce a very small p-value. Reporting the size of the effect alongside the p-value, not just whether it cleared 0.05, is what turns a statistical result into a practical one.
Questions people ask
Does a p-value of 0.03 mean there's a 3% chance the null hypothesis is true?
No, and this is the single most common misreading. A p-value of 0.03 means that, if the null hypothesis were true, there would be a 3% chance of seeing a statistic at least this extreme. It says nothing directly about how likely the null hypothesis itself is to be true, which is a different question that a p-value alone cannot answer.
Should I always use a two-tailed test?
Two-tailed is the safer default whenever a difference in either direction would count as meaningful, which covers most everyday comparisons. A one-tailed test is only defensible when the hypothesis genuinely rules out the opposite direction before the data is seen, not because it happens to produce a smaller p-value after the fact.
How do I know whether to use z, t or chi-square?
Use z when the population standard deviation is known or the sample is large; use t, with df usually equal to sample size minus one, whenever the standard deviation is estimated from the sample itself, which is the common case; use chi-square for goodness-of-fit tests, tests of independence, or tests about a single variance, where df is usually categories minus one.
Why does chi-square default to right-tailed?
The chi-square distribution only takes non-negative values and is not symmetric, so a large statistic, not a small one, is what points away from the null hypothesis in the standard goodness-of-fit and independence tests it is used for. Two-tailed chi-square tests exist but are unusual; selecting one here shows a note explaining that the p-value doubles the smaller tail mathematically rather than mirroring the shape of the curve.
Why is 0.05 the standard cut-off?
It is a convention suggested by R.A. Fisher in 1925 as a convenient round threshold, not a value derived from any underlying law. This calculator always shows the working at both 0.05 and 0.01 so a result can be read as a matter of degree rather than a single pass-or-fail line.
What happens with a very large test statistic?
The true p-value is never exactly zero, however large the statistic, but it can become far too small to show at four decimal places. Rather than displaying a misleadingly exact-looking 0.0000, the calculator shows "<0.0001" and notes that the underlying value is genuinely nonzero.
What is p-hacking?
It describes trying several reasonable-looking ways of analysing the same data and reporting whichever one crosses the significance threshold. Tried enough times, chance alone guarantees a small p-value will turn up somewhere even with no real effect, which is why deciding the analysis in advance matters more than distrusting any single result.
This calculator assumes the test statistic and degrees of freedom entered already come from a correctly specified test. It cannot check whether the underlying assumptions of that test, such as independence or approximate normality, actually hold for the data behind it.