What Is Effect Size?
Effect size describes how large a difference or association is, independent of sample size. It complements the p-value, which says whether there is statistical evidence against a null hypothesis but says nothing about magnitude. A large sample can produce a significant p-value for a trivial effect; a small sample can miss a substantial one. Effect size is the number that answers "how big."
Some effect sizes are standardized (Cohen's d, Hedges' g, r), which lets you compare effects across studies that used different measurement scales. Others are unstandardized (raw mean difference, risk difference), which keeps the effect in the original units. The right choice depends on the study design and outcome type.
Calculator Modes
The calculator supports eight design families: two independent groups, paired/repeated, one sample, from a t statistic, from ANOVA, from correlation, from binary outcomes, and conversion between measures. Each mode applies its own formula — the calculator does not run one generic effect-size formula on every design.
Cohen's d, Hedges' g, and Glass's Δ
For two independent groups, Cohen's d standardizes the mean difference by the pooled SD:
d = (M₁ − M₂) / sₚ , sₚ = √[((n₁−1)s₁² + (n₂−1)s₂²) / (n₁+n₂−2)]
The pooled SD weights each group's variance by its degrees of freedom. The sign of d follows M₁ − M₂; the magnitude is the effect size. With the calculator's default inputs (M₁ = 85, SD₁ = 10, n₁ = 30; M₂ = 78, SD₂ = 11, n₂ = 30), the mean difference is 7, the pooled SD is about 10.51, and d ≈ 0.67.
Hedges' g multiplies d by the small-sample correction J(df) ≈ 1 − 3/(4·df − 1). With large df the correction is negligible; with small samples (df under about 20) it makes a real difference. Hedges' g is generally preferred when reporting or pooling effects from small studies.
Glass's Δ divides the mean difference by a single reference-group SD instead of a pooled SD: Δ = (M₁ − M₂) / SD_ref. Use it when the treatment group's variance is expected to differ substantially from the control's — pooling would then mix two different scales. The calculator lets you choose which group's SD is the reference.
Paired and One-Sample Designs
Paired designs use a different effect-size definition based on within-pair differences:
d_z = mean(dᵢ) / SD(dᵢ) , dᵢ = afterᵢ − beforeᵢ
This is often called d_z or the repeated-measures standardized effect. It is not interchangeable with independent-groups Cohen's d — same letter, different denominator. The calculator reports the paired version separately and never substitutes one for the other.
One-sample designs compare a sample mean against a reference value: d = (M − μ₀) / SD. Hedges' correction applies the same way, with df = n − 1.
From a t Statistic
Converting a published t to an effect size depends on the design:
- Independent groups: d = t × √(1/n₁ + 1/n₂)
- One-sample or paired: d = t / √n
There is no single universal t-to-d rule. Applying a formula like 2t/√df across every t-test design produces wrong answers because the degrees of freedom do not encode the sample sizes each formula needs. The calculator asks for the design first, then applies the matching conversion. CLES is only computed for the independent-groups case.
ANOVA Effect Sizes
The ANOVA mode takes the sums of squares from an ANOVA table and computes:
- η² = SS_effect / SS_total — the proportion of total variability associated with the effect.
- Partial η² = SS_effect / (SS_effect + SS_error) — the proportion of variance associated with the effect after partialling out the error term.
- Cohen's f from η² = √(η² / (1 − η²)).
- Cohen's f from partial η² = √(partial η² / (1 − partial η²)).
The calculator reports both Cohen's f values separately because they are computed from different denominators and can differ in factorial or repeated-measures designs. η² and partial η² coincide only in one-way ANOVA; elsewhere they mean different things and should not be substituted for each other.
Correlation Effect Sizes
For correlation, r is the effect size and r² is the proportion of variance explained by the linear relationship. When you have two independent correlations, the calculator also computes Cohen's q = z(r₁) − z(r₂) using the Fisher z transform. q compares two correlations on the z scale — it is a comparison measure, not the same thing as r itself, and it only applies when the correlations come from independent samples.
Binary Outcomes
For binary outcomes the calculator reports:
- Risk difference (RD) = p₁ − p₀
- Risk ratio (RR) = p₁ / p₀
- Odds ratio (OR) = ((events + 0.5) / (non-events + 0.5))_treatment / ((events + 0.5) / (non-events + 0.5))_control
The OR always applies a 0.5 Haldane–Anscombe adjustment to all four cell counts, which keeps the ratio finite if any cell is zero. RD and RR are computed on the raw proportions. log(RR) and log(OR) are also reported. These measures describe risk and odds — they are not standardized mean differences and should not be labeled as Cohen's d.
CLES and Effect-Size r
Two additional measures appear alongside d in the independent-groups mode and the from-t independent design:
- Effect-size r = d / √(d² + 4). This is a correlation-style representation of d. It is a model-based conversion, not a Pearson correlation computed from raw paired observations.
- CLES = Φ(d / √2). The Common Language Effect Size expresses the comparison as a probability — the probability that a randomly chosen observation from Group 1 exceeds a randomly chosen observation from Group 2, under a normal equal-variance model. The direction of comparison depends on which group is labeled Group 1.
Neither measure adds sample information. Both repackage the same d under different assumptions. The conversion mode also computes CLES when converting from d or r.
Confidence Intervals
For d in independent-groups, paired, and one-sample modes, the calculator reports an approximate standard error and a confidence interval at 90%, 95%, or 99%. The SE uses a Hedges–Olkin-style approximation; the interval assumes approximately normal sampling of d. These intervals are adequate for typical sample sizes but less reliable in small samples with unequal variances, where a bootstrap or robust estimator would be preferred.
Interpretation and Benchmarks
Cohen's conventional benchmarks — 0.2 small, 0.5 medium, 0.8 large — are rough orientation values, not universal definitions of practical importance. A d of 0.5 in one research area might be trivial; in another, 0.3 might be important. The sign indicates direction; the magnitude describes standardized distance. Report both alongside the design, the measurement scale, and the practical context.
Effect Size vs Statistical Significance
A p-value answers whether the data provide evidence against a null hypothesis. Effect size answers how large the observed effect is. A very large sample can produce a significant p-value for a d of 0.05; a small sample can miss a d of 1.0. Both are useful, but they answer different questions, and reporting one without the other leaves the picture incomplete.
Comparison Table
| Measure | Typical use |
|---|---|
| Cohen's d | Independent-groups standardized mean difference |
| Hedges' g | Small-sample corrected standardized mean difference |
| Glass's Δ | Standardized difference using a reference-group SD |
| d_z | Paired standardized effect (within-pair differences) |
| Effect-size r | Correlation-style representation of d |
| η² | ANOVA proportion of total variance |
| Partial η² | ANOVA proportion after partialling out error |
| Cohen's f | Standardized ANOVA effect |
| Cohen's q | Difference between two Fisher-z correlations |
| CLES | Probability-based group comparison |
| RD / RR / OR | Binary outcome measures |
Common Mistakes
- Reporting a p-value without an effect size, or an effect size without a p-value.
- Using independent-groups d on paired data (or vice versa). The denominators are different.
- Calling every standardized difference "Cohen's d." Hedges' g, Glass's Δ, and d_z use different standardizers.
- Confusing η² with partial η². They coincide in one-way ANOVA but diverge in factorial and repeated-measures designs.
- Applying a single t-to-d formula across every t-test design. Independent and one-sample/paired need different conversions.
- Treating 0.2 / 0.5 / 0.8 as universal thresholds.