Chapter 13 Exploring heterogeneity
This chapter is heavily based on my in-press, open access book chapter “Small sample meta-analyses: Exploring heterogeneity using MetaForest”. The book does not yet have a DOI, but information about it will be anounced here:
A valid reference for the methods described in this chapter is Van Lissa (2017).
In the social sciences, meta-analyses often pool research conducted in different laboratories, using different methods, instruments, and samples. Such between-studies differences can introduce substantial heterogeneity in the effect sizes found. At the same time, the sample of studies is often small, which means there is limited statistical power to adequately account for moderators that cause between-studies heterogeneity. If we just include all moderators in a meta-regression, we risk overfitting the data. In this chapter, I introduce a technique that can explore between-studies heterogeneity and perform variable selection, identifying relevant moderators from a larger set of candidates, without succumbing to overfitting: MetaForest, a machine-learning based approach to identify relevant moderators in meta-analysis (Van Lissa 2017).
References
Van Lissa, Caspar J. 2017. “MetaForest: Exploring Heterogeneity in Meta-Analysis Using Random Forests.” Open Science Framework, September. https://doi.org/10.17605/OSF.IO/KHJGB.