Chapter 4 Calculating Effect Sizes

Papers do not always report the effect sizes exactly the way you want to meta-analyze them. This chapter addresses the basics of calculating effect sizes.

Meta-Analysis requires an effect size and an estimate of the sampling variance of that effect size for each study. Papers do not always report the effect size, or they report a different effect size than the one you want to use in your meta-analysis. This chapter addresses the basics of calculating effect sizes.

It may not immediately be obvious whether a paper reports the necessecary statistics to calculate an effect size. You will need an ‘effect size’ and its sampling variance. As a general guideline, ask yourself these questions:

  1. Am I meta-analyzing a descriptive statistic (mean, SD, proportion, Cronbach’s alpha), a measure of association (correlation or bivariate regression coefficient), or a difference (e.g., mean difference)?
  2. Is that statistic reported directly?
  3. Is its variance or SE reported directly?
  4. Do I have the sample size for the total group or for each group I’m comparing?
  5. Are measures of variability reported (e.g., SD for each group)?

If you cannot figure out whether you have sufficient information to calculate the effect size, I recommend contacting a statistician.

Researchers can get quite creative in trying to obtain the relevant information. It is expected that researchers contact authors of papers with incomplete information to request that information. Many journals require the authors to provide this information upon request. Moreover, researchers sometimes use an on-screen ruler (e.g., https://www.arulerforwindows.com/) to measure means and SEs from graphs, if these are not reported in the text of the paper.