It is not always possible to conduct RCT for impact evaluation in the actual world. As a result, evaluators explore other plausible choices to build on the counterfactual utilizing a quasi-experimental design(QED) approach. As the name implies, quasi-experiment design (QED) is similar to experimental design, including creating counterfactuals utilizing other likely options. In the case of quasi-experimental design, evaluators attempt to reduce confounding factors by constructing a comparison group and, to the greatest extent possible, mirroring the qualities of the control group. In the case of quasi-experiment design, identifying a comparison group and selecting a robust counterfactual become crucial in attributing changes in outcomes and impact concerning specific treatments.
Using quasi-experimental design
Using a quasi-experimental design instead of a real experimental design may be advantageous for ethical or practical grounds. In some circumstances, adopting a random assignment in the study might be unethical, for example, when one treatment group receives the intervention and another does not. In a quasi-experimental study, you can investigate a causal relationship without restricting others from receiving the intervention.
Further, there are other instances wherein one may not have the luxury of resources one needs to roll out an experiment design but still want to ascertain the causality; QED design can be an effective alternative. Practical scenarios also determine the use of QED design, wherein in real-world situations, its not always feasible to minimize the contamination or other practical considerations make it inevitable for intervention to scale up; in such situations, a QED design could be the best bet for evaluators to asses the attribution.
Quasi-experimental design types #
These are the most typical quasi-experimental designs:
- Non-equivalent groups design: Participants in this design take a pre-test and post-test to determine cause and effect.
- Regression discontinuity design: Regression discontinuity design uses the propensity score of a pre-treatment variable to assign participants to a certain treatment randomly.
- Interrupted time series design: Using this method, researchers follow individuals before and after the intervention for a protracted period.
Examples of quasi-experimental design #
In addition to the broad typology mentioned above, the specific popular design types employed in QED are:
- Difference in Difference
- Propensity Score Matching
- Regression Discontinuity
- Instrumental Variable
- Interrupted Time Series Design
Advantages of a quasi-experimental design #
- Because they are conducted in naturalistic situations, quasi-experimental designs are more likely to have more external validity than experiments. It means the findings are more likely applicable in real-world circumstances.
- QED designs are more practical as they try to mimic real-world situations and can adapt to changing implementation and policy environments than experiment.
- QED designs are usually less expensive and easier to carry out than quasi-experimental designs.
Disadvantages of a quasi-experimental design #
- In comparison to real experiments, it serves less internal validity.
- In QED, one cannot be certain that the confounding or third variable is eliminated because there was no randomization.
- In QED design, sometimes it becomes messy to assess the attribution because of non-balanced project and comparison groups.
