In impact evaluation, not every question can be answered by setting up a controlled experiment. Sometimes the variable to be studied can’t be manipulated, or doing so wouldn’t be ethical or practical. This is where a non-experimental research design comes in.
What is non-experimental research? #
Non-experimental research design studies people, programmes, or events as they naturally occur. There’s no manipulation of an independent variable and no random assignment of participants to groups. Researchers observe, measure, and interpret existing conditions instead of creating new ones.
Characteristics of non-experimental research #
- Groups already exist; they aren’t created by the researcher.
- The researcher doesn’t intervene in the environment being studied.
- Findings describe associations, not proven cause-and-effect.
- It often draws on surveys, case studies, or administrative records.
Non-experimental quantitative research still uses numbers and statistical analysis to identify patterns and correlations. The difference lies in what those numbers can tell — relationships, not causation.
When to use a non-experimental design? #
This approach works well for exploratory questions, single-variable studies, or research where manipulating the variable isn’t feasible, for instance, studying the effects of poverty or migration, where randomly assigning people to conditions simply isn’t possible.
Non-experimental research design vs experimental research design #
Experimental designs manipulate variables and randomly assign participants, which strengthens causal claims. Non-experimental designs trade that certainty for flexibility, lower cost, and the ability to study real-world settings as they actually unfold.
For evaluators, the choice rarely comes down to which design is inherently “better.” It comes down to the question being asked, the resources on hand, and what’s realistic and ethical to measure in the field. In many programme evaluations, non-experimental approaches end up being the only honest way to learn something useful about how change is unfolding on the ground.
List of recommended resources #
For a broad overview #
7.1 Overview of Non-Experimental Research
This tutorial by Dr. Danielle Graham gives an overview of non-experimental research and distinguishes it from experimental research, giving several examples. Dr. Graham also explains when a researcher might choose to conduct non-experimental research instead of experimental research.
Quantitative Research with Nonexperimental Designs
This post by Dr. Janet Salmons gives an overview of quantitative research, a type of non-experimental design, and provides various resources and articles along with case studies for further research on the topic.
For in-depth understanding #
6.1 Overview of Non-Experimental Research
This chapter from Research Methods in Psychology gives a broad understanding of non-experimental research, when to use it, and its different types. It also revises internal validity and how it is applicable to non-experimental research.
Quantitative Research for the Qualitative Researcher
Chapter 7 of this book by Laura M. O’Dwyer and James A. Bernauer focuses specifically on non-experimental research designs, discussing its characteristics, its strengths and limitations, as well as the steps to conduct non-experimental research.
Case study #
This World Bank study uses a non-experimental impact evaluation to assess the outcomes of a large-scale rural roads project in Vietnam. The findings suggest that combining transport investments with complementary development programmes can help ensure that poorer and female-headed households benefit more equitably.
This paper by John Gibson, David McKenzie, and Steven Stillman studies the income gains from migration by comparing migrants from Tonga to New Zealand with lottery applicants who were not selected. It also evaluates how accurately non-experimental methods estimate migration outcomes. The findings show that these methods often overestimate income gains, although approaches such as instrumental variables and propensity score matching yield more reliable estimates.
References #
Non-experimental research: What it is, overview & advantages
Non-Experimental Research Designs: Amenable to Nursing Contexts
