Impact evaluation is essentially about answering a simple but important question: Did this programme actually make a difference? To get a reliable answer, evaluators use different methods, each offering a slightly different lens. Instead of relying on one rigid approach, good evaluations often blend methods to reflect real-world complexity.
Qualitative and Quantitative Methods #
Most impact evaluations begin with a choice between qualitative and quantitative methods or a combination of both.
- Quantitative methods
They deal with numbers. They track changes in things like learning levels, income, or access to services and help show whether outcomes improved.
- Qualitative methods
These focus on people’s stories and experiences, using interviews, group discussions, or observations to understand how those changes were felt on the ground.
Together, they move the analysis beyond just figures and percentages.
Experimental and Quasi-Experimental Designs #
To clearly link outcomes to an intervention, evaluators often turn to experimental and quasi-experimental designs.
- Randomized control trials in development involve placing participants into programme and non-programme groups by chance, which helps evaluators see whether observed changes can truly be traced back to the intervention rather than to other factors.
- When randomization isn’t practical or ethical, quasi-experimental designs step in.
One commonly used technique is the difference-in-differences method, which compares how outcomes change over time for groups that did and did not receive the intervention.
Mixed-Method Impact Evaluation #
Mixed-method impact evaluation brings together qualitative and quantitative methods in a deliberate way. Numbers show the scale of change, while qualitative insights explain what worked, what didn’t, and why, making findings more grounded and useful.
Theory-Based Evaluation Approaches #
Rather than asking only whether a programme worked, theory-based evaluation approaches dig into how it worked. By testing assumptions in the theory of change, these approaches reveal the pathways through which impact unfolds.
In Technique
There is no single “best” impact evaluation method. The strongest evaluations choose approaches that fit the context, the question, and the people involved, producing evidence that is both credible and meaningful.
List of recommended resources #
For a broad overview #
How many types of impact evaluation are there?
This video tutorial by M&E Guru on YouTube provides an introductory understanding of the different types of impact evaluation. It discusses experimental impact evaluation, quasi-experimental impact evaluation, and non-experimental impact evaluation.
This handout by The Abdul Latif Jameel Poverty Action Lab provides an overview of the the different methods of impact evaluation such as randomized controlled trials, simple difference, differences in differences, among others,
For in-depth understanding #
These impact evaluation approaches, as outlined by the UNDP Independent Evaluation Office, focus on identifying changes that can be directly linked to a particular programme or intervention. They play a crucial role in helping organizations make evidence-based decisions about whether an intervention should be continued, adapted, scaled up, or discontinued.
Impact Evaluation Methodologies
This slide deck created by Martin Abel for The Abdul Latif Jameel Poverty Action Lab provides an in-depth explanation of impact evaluation and its methodologies. Abel uses case studies and examples to illustrate and explain the methodologies of impact evaluation.
Case study #
This study adopts an explanatory mixed-methods design, combining qualitative inquiry with a quasi-experimental quantitative approach. It examines the causal effects of linking supply-side performance-based financing incentives with demand-side cash transfers on access to maternal and neonatal healthcare services, as well as on the quality and equity of care in Malawi.
This impact evaluation examines an intimate partner violence prevention programme in Rwanda through a randomized controlled trial design. The study assesses a 22-week couples’ training intervention focused on strengthening communication, challenging gender norms, and encouraging more equitable relationships.
References #
Impact Evaluation: Definition, Methods & Best Practices
What Is Impact Evaluation? (Benefits, Types, and Planning Tips)
