Introduction to Meta-Analysis

Sambodhi > Blog > Monitoring and Evaluation > Introduction to Meta-Analysis
Posted by: Kultar Singh
Category: Monitoring and Evaluation
Meta-Analysis

Meta-analysis, as the name suggests, is the analysis of the way studies are analyzed. This process is carried out methodically, according to criteria, and includes a data pool based on quantitative analysis. Such a review identifies key findings and trends that may impact future research and policy choices. The main goal of meta-analysis is to produce a summary of combined data that are examined to identify differences. This helps evaluate effects in various subgroups, develop new theories to guide future research, overcome the drawbacks of limited sample numbers, and establish statistical significance.

While doing a meta-analysis, researchers look for similarities in several studies’ results to answer the research topic. A meta-analysis often investigates the association between one dependent variable and one independent variable. There is a wide variation in the magnitude of the association across research, with some studies showing a significant degree of relationship and others showing no relationship at all. In such instances, it is prudent to arrive at an overall estimate by aggregating the findings of several investigations in advance if the need arises. 

During meta-analysis, researchers must formalize the association they discover in the initial step of the process. Following the development of a hypothesis, they must compile all relevant studies to provide information about the same. Consequently, they must code all of the studies that have been compiled to determine the effect size. Researchers should then examine the distribution of effect sizes to determine the nature of the association.

Studies with poor design must be excluded from meta-analysis. If such studies become part of the sample, the weights allocated to these studies should be different from those assigned to research with good design to prevent misleading findings. 

A meta-analysis collects statistical approaches that aggregate information from several studies to forecast an effect estimate. It is a method of combining several sets of findings to provide an overall conclusion or an estimate. Scientists can compare the effect sizes obtained by two separate studies with one another by using the following formula:

 1/2

 = (z1 – z2)/[(1/n1 – 3) + (1/n2 – 3)]

For example, z1 and z2 are defined as the Fisher transformations of r, while the sample sizes for n1 and n2 are defined as the sample sizes for each study, respectively.

Researchers may even do meta-analyses with the use of statistical tools. It begins by compiling a database of papers, after which it carries out a meta-analysis to arrive at an overall estimate.

Advantages and Disadvantages 

Meta-analysis is also often used in fundamental research to assess evidence in sociology, social psychology, finance and economics, political science, marketing, and ecology. Further, several journals now encourage academics to publish systematic reviews and meta-analyses that synthesize the body of data on a certain topic. These are frequently used as review mechanisms to help build other studies. Meta-analyses may help researchers design future experiments. They also help determine which questions have already been addressed and which ones still need to be answered.

Compared to individual research, meta-analyses provide several benefits as they have stronger statistical power and extrapolate a larger population. Meta-analyses may also demonstrate statistical significance between research that seems to have contradictory findings. This is important because statistical significance strengthens the validity of any differences found and improves the information validity. 

While there are benefits to meta-analysis, it is time-intensive and requires advanced statistical abilities and methodologies. Further, in the case of meta-analysis, one should also safeguard from bias that might arise from publication bias (where a certain kind of study is more likely to be published) or selection bias ( when the researcher does not clearly articulate the reasons for selecting the studies).

Meta-Analysis and a Systematic Review

Systematic review and meta-analysis are two closely related concepts but are distinct from one another. In science, systematic reviews are a kind of scientific research aimed at a specific issue with certain elements. These elements include well-defined, planned techniques for locating and choosing relevant studies and reviewing and synthesizing the findings of related but separate studies. A quantitative synthesis of the data from a large number of research may or may not be provided by this method. While doing a meta-analysis, it is necessary to statistically combine the data from similar research papers identified via a systematic review.

References: 

Card, N. A. Applied meta-analysis for social science research. Guilford, New York, 2012.

Cooper, Harris M. Research synthesis and meta-analysis: A step-by-step approach, 5th EditionThousand Oaks, CA: Sage, 2016.

Crocetti, Elizabetta. “Systematic reviews with meta-analysis: Why, when, and how?” Emerging Adulthood, 4 (1), 2016. pp. 3-18.

Dickersin, K. “Publication bias: Recognizing the problem, understanding its origins and scope, and preventing harm. In H. R. Rothstein, A. J. Sutton, M. Borenstein (Eds.)” Publication bias in meta-analysis. Chichester: John Wiley & Sons, Ltd, 2005, pp. 127-144

Hedges, L. V., & Pigott, T. D. “The power of statistical tests for moderators in meta-analysis”. Psychological Methods, 9, 2004. pp. 426–445.

Higgins, J., & Thompson, S. G. “Quantifying heterogeneity in a meta-analysis”. Statistics in Medicine, 21,2002. pp. 1539-1558.

Hunter, J. E., & Schmidt, F. L. Methods of meta-analyLsis: Correcting Error and Bias in Research Findings, 3rd Edition. Newbury Park, CA: Sage Publications, 2014. 

Kultar Singh – Chief Executive Officer, Sambodhi

Author: Kultar Singh

Leave a Reply