The impact of migration on the welfare of households left behind in rural ghana. The evaluation is funded through research grants from the soros foundation and the international growth centre. Evaluate your program and demonstrate program effectiveness. This is always the case for expost impact evaluation designs. Quasiexperimental research designs are the most widely used research approach employed to evaluate the outcomes of social work programs and policies. Timing of intervention and data collection for impact evaluations with a. Program participants themselvesbefore participating in the program. Evaluation designs slides corporation for national and. Use of quasiexperimental designs for vaccine evaluation. In a quasiexperimental approach nonrandom methods of assignment are used. A, crosssectional, controlled design, where the impact equals the difference d 1 between the study group black circle and control group gray circle. Quasiexperimental designs identify a comparison group that is as similar as possible to the intervention group in terms of baseline preintervention characteristics. We carried out a quasi experimental impact evaluation of social impacts of wmas, collecting data from 24 villages participating in 6 different wmas across two geographical regions, and 18.
Unless there was random assignment then a quasi experimental design has to be used. Regression discontinuity design technical track impact evaluation workshop gdansk, february 2017. It may also be necessary to use quasi experimental designs for exante impact evaluations, for example, where ethical, political or logistical constraints, like the need for a phased geographical rollout, rule out randomization. This document provides an example of a detailed evaluation plan for evaluating the. A quasi experiment is an empirical interventional study used to estimate the causal impact of an intervention on target population without random assignment. Having a manual that covers sample size calculations for quasiexperimental designs would also be useful. Seasonal malaria chemoprevention smc for children pdf 834. The place impact evaluation comprised two quasi experimental designs to determine whether the implementation of oakland ceasefire was associated with citywide reductions in gun homicide. Both experimental and quasiexperimental designs can produce credible impact evaluation findings, but there is a difference, and their classifications signal what that difference is. Quasiexperimental evaluations part 6 in a series on practical evaluation methods kristin anderson moore, ph. Overview although experimental studies are described as the gold standard for assessing the effectiveness of a program in changing outcomes, in some cases, quasiexperimental studies may be more feasible or appropriate. Experimental studies try to determine causality or correlation in so far as this is possible.
Experimental impact evaluation 4 evaluative research. Attention will also be given to issues related to the measurement of outcomes, ensuring detection of meaningful program effects, and interpreting. A quasiexperimental controlled evaluation of the impact. Purpose of impact evaluation impact evaluation serves both objectives of evaluation.
Developed using the evaluation plan template, the plan is for a quasi experimental design qed. The criminal justice pilot paralegals solicit criminal cases through several strategies, including regular. The impact study will used a quasiexperimental design to assess the impact of. This convergence of rising need for impact evaluation evidence among. Example evaluation plan for a quasiexperimental design. Experimental and quasiexperimental evaluations 41 hannum. Along this continuum, the preference in usaids evaluation policy is clear. Your programs theory of change and logic model primary purpose of the evaluation and key research questions. Applicability of impact evaluation european commission. Sociologists and psychologists often perform quasiexperimental research to. Quasi experimental methods that involve the creation of a comparison group are most often used when it is not possible to randomize individuals or groups to treatment and control groups.
However, care must be taken about generalizing from a specific context. Appropriate evaluation methods therefore depend on several overall criterion. Often it is hard to recognize or justify spending on evaluation when it does not directly impact the programs community or has a large lagtime. Quasiexperimental designs typically allow the researcher to control the assignment to the treatment condition but using some criterion other than random assignment e. An impact evaluation design prepared once the intervention has started, and possibly been completed. Outline of principles of impact evalation discussion dra. C, controlled prepost design, in which the impact equals the.
This means there may be systematic differences between the groups. A quasiexperimental impact evaluation joseph kofi teye, louis boakyeyiadom, edward asiedu, julie litchfield, johnson wilson appiah kubi and mariama awumbila working paper 62 november 2019. Both experimental and quasi experimental designs can produce credible impact evaluation findings, but there is a difference, and their classifications signal what that difference is. Quasiexperimental impact evaluation expost of ifads ad2m project in madagascar final report august 2017 hannah ring, american institutes for research mitchell morey, american institutes for research erin kavanagh, american institutes for research kevin kamto, american institutes for research nancy mccarthy, lead analytics. Quasi experimental designs identify a comparison group that is as similar as possible to the intervention group in terms of baseline preintervention characteristics. Experimental, quasiexperimental or a descriptive and study. Therefore evaluative research remains an underutilized tool.
Quasi experimental research shares similarities with the traditional experimental design or randomized controlled trial, but it specifically lacks the element of random assignment to treatment or control. Excerpt quasi experimental research designs, like experimental designs, test causal hypotheses. Quasiexperimental better evidence methods in action. A matchedcomparison group design allows the evaluator to make causal claims about the impact of aspects of an intervention without having to. Quasiexperimental study comparing the proportion of swaps made by an intervention group 267 families who had signed up to the smart swaps campaign promoted through various media, including television and radio advertising in early 2014 and a comparison group 5 families resident in wales, signed up for change4life materials, but not directly exposed to the smart. A quasi experimental impact evaluation joseph kofi teye, louis boakyeyiadom, edward asiedu, julie litchfield, johnson wilson appiah kubi and mariama awumbila working paper 62 november 2019.
Effects of a food hub initiative in a disadvantaged community. Evaluation theory, design and methods tei the evaluators. It may also be necessary to use quasi experimental designs for exante. However, in a quasi experimental approach the target population is not randomly allocated to the intervention and comparison groups. A quasiexperimental design is one that looks a bit like an experimental design but lacks the key ingredient random assignment. In conclusion, quasiexperimental designs represent a useful tool for public health evaluation, as well illustrated in the article by harper and colleagues. Quasiexperimental approaches are similar in that they compare units that are part of a development intervention with those that are not. Evaluation design is the structure that provides the information needed to answer each of your evaluation questions.
If the program did not exist, outcomes would be the. Quasi experimental studies are research designs that evaluate human behavior. Quasiexperimental impact evaluation of ifads ad2m project. In conclusion, quasi experimental designs represent a useful tool for public health evaluation, as well illustrated in the article by harper and colleagues. These flexible methods can be adapted to a wide range of public health evaluation studies, and can incorporate assumptions and previous knowledge on the association of interest. A quasi experimental design is an empirical study, almost like an experimental design but without random assignment. Power calculation for causal inference in social science. My mentor, don campbell, often referred to them as queasy experiments because they give the experimental purists a queasy feeling. Methods of data analysis used in quasiexperimental designs may be expost single difference or. An impact evaluation relies on rigorous methods to determine the changes in outcomes which can be attributed to a specific intervention based on causeandeffect analysis. Impact evaluation designs program cycle project starter.
The comparison group captures what would have been the outcomes if the programmepolicy had not been implemented i. Quasiexperimental design and methods better evaluation. A quasiexperimental controlled evaluation of the impact of a hospice rapid response community service for endoflife care on achievement of preferred place of death laura m holdsworth, heather gage, simon coulton, annette king, and claire butler. Quasiexperimental methods require more assumptions randomized assignment differencein. White, h and sabarwal, s 2014 quasiexperimental design and methods, methodological briefs. Ie technical note united states agency for international. Today, many states are revisiting their use of this ban, searching for ways to decrease the size of their prison populations. Your intended evaluation design should be based on and aligned with the following.
A quasiexperimental impact evaluation download wp62teyeetal2019theimpactofmigrationonthewelfareofhouseholdsleftbehindinruralghana. In this brief, we describe varied types of quasiexperimental evaluations and the. An evaluation may examine the impact of an intervention on crime levels or whether an intervention has changed the perception of individuals. A quasiexperimental evaluation of the impact of public. The place impact evaluation comprised two quasiexperimental designs to determine whether the implementation of oakland ceasefire was associated with citywide reductions in gun homicide. This document provides an example of a detailed evaluation plan for evaluating the effectiveness of an intervention. Effects of a food hub initiative in a disadvantaged. My mentor, don campbell, often referred to them as queasy experiments because they give the experimental purists a. A quasiexperiment is an empirical interventional study used to estimate the causal impact of an intervention on target population without random assignment. What is meant by evaluating the impact of a program. A matchedcomparison group design allows the evaluator to make causal claims about the impact of aspects of an intervention without having to randomly assign participants. A quasiexperimental study to assess the impact of four adolescent sexual health interventions in subsaharan africa context. Rigorous evaluations are needed to assess whether adolescent sexual health interventions have an effect on young peoples riskrelated perceptions and behaviors.
External validity the extent to which the results of the impact evaluation apply to another time or place. Having a manual that covers sample size calculations for quasi experimental designs would also be useful. In this paper, we discuss what a quasi experimental design is, when they might be used for vaccine evaluation, their strengths and limitations, and examples of their application. Experimental and quasi experimental study designs are often utilized in summative evaluations. The basic structure of a quasiexperimental evaluation involves examining the impact of an intervention by taking measurements before and after it is implemented. Quasiexperimental designs chris fifeschaw contents 4. Report of a study commissioned by the department for. Four evaluations were based on randomized controlled trials, and six on nonrandomized, quasiexperimental designs. Developed using the evaluation plan template, the plan is for a quasiexperimental design qed. Although we focus on vaccination, the methods described can equally be applied to evaluations of other infectious disease interventions that act at a population level. Excerpt quasiexperimental research designs, like experimental designs, test causal hypotheses. Introduction to impact evaluation design usaid learning lab. Impact evaluation is an assessment of how the intervention being evaluated affects outcomes, whether these effects are intended or unintended. Quasiexperimental design research methods knowledge base.
Quasi experimental methods can be used retrospectively, i. Quasi experimental research designs are the most widely used research approach employed to evaluate the outcomes of social work programs and policies. The welfare act of 1996 banned welfare and food stamp eligibility for felony drug offenders and gave states the ability to modify their use of the law. For an evaluation to have internal validity, the outcome of the control group needs to be indeed a. These methods can be used to attribute change at any program or project.
The impact of migration on the welfare of households left. Quasiexperimental design and methods unicef innocenti. Quasiexperimental methods that involve the creation of a comparison group are most often used when it is not possible to randomize individuals or groups to treatment and control groups. That is, obtain evidence that the program caused the outcomes observed in beneficiaries. Before applying either experimental or quasi experimental methods, evaluators must lay out the. Mar 09, 2016 the webinars were based on the impact evaluation series a userfriendly package of methodological briefs and four animated videos and presented by the briefs authors. Quasi experimental designs typically allow the researcher to control the assignment to the treatment condition but using some criterion other than random assignment e. This study evaluates our mentored training program in terms of scientific productivity among applicants. Before applying either experimental or quasiexperimental methods, evaluators must lay out the.
A quasiexperimental design is an empirical study, almost like an experimental design but without random assignment. A quasiexperimental impact evaluation of a criminal. Mentored training and its association with dissemination. Quasiexperimental studies are research designs that evaluate human behavior. Experimental and quasiexperimental designs involve comparisons. Pdf impact evaluation of seasonal malaria chemoprevention. B, prepost design, in which the impact equals the difference d 1 between pre and postintervention observations. For impact evaluations, experimental methods generate the strongest evidence. Rigorous impact evaluations may be undertaken for development objectives. Improving the rigor of quasiexperimental impact evaluations. Randomisation and quasiexperimental designs that establish control. A quasiexperimental study to assess the impact of four.
The program was the only factor influencing changes in the outcome over time. This document is part of the better evidence in action toolkit. Impact evaluation designs are identified by the type of methods used to generate the counterfactual and can be broadly classified into three categories experimental, quasiexperimental and nonexperimental designs that vary in feasibility, cost, involvement during design or after implementation phase of the intervention, and degree of. Impact evaluations are characterized by a specific evaluation design quasiexperimental or experimental in order to answer a causeandeffect question. Sociologists and psychologists often perform quasi experimental research to study individuals within the same environment.
These designs are often referred to as patchedup research designs poister, 1978, and usually, they do not test all the causal linkages in a logic model. It may also be necessary to use quasiexperimental designs for exante. Quasiexperimental research shares similarities with the traditional experimental design or randomized controlled trial, but it specifically lacks the element of random assignment to treatment or control. Impact evaluations need to account for the counterfactual what would have occurred without the intervention through the use of an experimental or quasiexperimental design using comparison and treatment groups.
586 767 39 1263 655 550 636 388 952 651 764 153 1389 866 185 118 405 304 1268 455 196 147 1003 1532 1362 841 291 69 467 126 1293 117 150 658 381 423 726 913 944