1. Furthering climate change adaptation, Impact of DFID rural programs in India.
In developing countries, a large part of the livelihood portfolio comprises of natural resources and ecosystem services, and these are critical for sustainable livelihoods. It has been universally acknowledged that climate change enhances risks of those dependent on soil, water, forests and other natural resources for their livelihoods. With similar considerations in the Indian context, adaptation strategies have to take cognizance of, and integrate livelihood protection and enhancement as integral components. Poverty being both the condition as well as a determinant of vulnerability, poverty reduction is imperative for any resilience building exercise and a starting point for such an exercise is to understand assets and capabilities of the poor. With the poor being dependent on ecosystem goods and services for their livelihoods, Natural Resources Management enhances the quality and sustainability of ecosystem services as well as supports livelihoods generation. This in turn increases coping capacities and resilience of the vulnerable to extreme events and related stresses.
Although DFID’s rural livelihoods programmes in India have varying approaches, they have a unifying goal of reduction of rural poverty through promotion of sustainable livelihoods. Though, not designed with climate change adaptation interventions, the Sustainable Livelihoods Framework with inherent considerations of the Vulnerability Context that have direct impact on people’s assets as well as livelihood options are enhancing resilience of communities for coping with “shocks”, “trends” and “seasonal shifts”.
This paper is based on synthesis of findings of Impact Assessment of two of the DFIDs rural projects in India namely Western Orissa Rural Livelihoods Project and Madhya Pradesh Rural Livelihoods Project. The paper analyses the findings on enhancement of livelihood assets for enhanced ability for withstanding climate change vulnerabilities. Though inherently livelihoods protection and enhancement interventions, with the evidenced results, these rural interventions essentially are “win-win” adaptation options.
2. CRISP framework for post-project evaluation
Development projects do not continue for infinite duration. Funding and implementing agencies withdraw from the program area after a certain point of time. Phasing out of programs is a critical phase fraught with several challenges. Most of the agencies and donors do not have a mandate for post-project evaluations and therefore commit little or no funds for post-project evaluations. Post-project evaluation examines if the project has led to sustainable outcomes and practices in the community. The paper outlines relevant case studies and explores challenges associated with post-project evaluations. It proposes a framework to address the envisioned objectives of post-project evaluation encapsulating sustainability of Community, Resources, Institutional and Processes (CRISP). It further suggests Difference-In-Difference research design for conducting post-project evaluation.
3. Propensity score Matching method in quasi experimental designs, an approach to program evaluation of INHP-II
The experimental designs are generally considered as the robust evaluation methodologies as there is random assignment. These are possible in clinical trials or in pilot phase of the project but during the development phase due to ethical issues and resource constraints; use of true experimental designs are not feasible in majority of development interventions as use of experimental design entails creation of treatment and comparison group thereby providing benefits to some and excluding others. It is unethical at program-level to provide the benefits to few and leave others and thus, there is difficulty in construction of both treatment and comparison at baseline. This makes attribution of observed outcomes and impacts to program intervention very difficult. The task gets more difficult when there are no baseline studies available. PSM offers one such alternative for addressing the concerns comparison and attribution. This paper is based on the case of Endline Evaluation of INHPIII where the Quasi-Experimental Design was employed using the PSM technique to construct the ideal comparison match for the treatment groups.
4. Rigorous Integrated Generalized (RIG) Qualitative Assessment
Qualitative research in development sector is an important paradigm which is used standalone or in conjunction with quantitative research. There is quantitative-qualitative paradigmatic divide on the basis of ontology, epistemology and methodology which explains the strengths and limitations of both paradigms. However, the significance of pragmatism in research sector drags qualitative paradigm under criticism as the traditional notions of validity and reliability do not fit in this paradigm. This paper attempts to instil rigor in qualitative research and uses Integrated Generalization design to derive both theory and generalizable result. On this ground, the paper proposes Rigorous Integrated Generalized (RIG) model for qualitative assessment.
5. A framework for Monitoring and evaluation of Climate change adaptation interventions.
Climate change has become one of the most important global issues of our time, with far-reaching natural, socio- economic, and political impacts. In order to equip the community to deal with the effects of climate changes, various adaptation interventions have been furthered. However, efficacy of these interventions varies in terms of their ability to address specific climate change vulnerabilities of human populations and the natural and economic systems. To understand the efficacy of the interventions towards envisaged climate change results, rigorous monitoring and evaluation of these interventions becomes imperative both for ensuring efficiency, results, cost-effectiveness and sustainability of the interventions. With these considerations, programme logic model can be an appropriate overarching.
Monitoring & Evaluation Framework. This paper takes programme logic model as the starting point and describes key principles that need to be factored in developing a monitoring and evaluation framework for climate change adaptation projects. The projects draws upon good practices of various adaptation interventions across the globe to propose established guiding principles.
6. Mixed-Design Approach in Impact Evaluation: Principles and Practice
There is no single method in impact evaluation that can always address the different aspects better than others. Importance of mixed design approach in impact evaluation studies arises with the need for attribution that cannot always be addressed through quantitative approach. ‘Mixed-methods’ is the combination of qualitative and quantitative approaches that help in outlining the causal chain and firming up a clear identification strategy in impact evaluation. When used in combination, both quantitative and qualitative data yield a more complete analysis, and they complement each other. Sometimes the evaluation design may emerge in new ways, depending on the conditions and information that is obtained during the study. The present paper discusses the application of mixed design approach in light of evaluation of Title II Safe Motherhood and Child Survival (SMCS) programme of Catholic Relief Services (CRS). The programme employed both the traditional quantitative and the qualitative methods to evaluate the impact of the intervention. Whereas the outcome level indicators were calculated using the quantitative methods, the strengthening of institutional and community level processes and sustainability linkages were derived at through qualitative tools.
7. A parable of discrimination and Vignettes
Discrimination is a sensitive topic. Conventional research tools like household schedules and Focus Group Discussions have certain drawbacks in assessing discrimination. Vignettes offer a less threatening way to explore sensitive topics like discrimination. This paper gives a step-by-step account of assessing discrimination in social programs using Vignettes. The paper adopts a fictional approach and finds an enterprising social researcher in the main protagonist: Ms. Inquisitive: A social researcher. Over the course of the story, Ms. Inquisitive, explores using Vignettes as one of the effectual tools to assess discrimination in social programs. She learns about the challenges and complexities faced while using Vignettes as a research tool. She meets several protagonists and learns about different types of Vignettes, methods of drafting and analysing Vignettes. She also gets an insight into findings from the field, where one of the characters administers the research tools and shares his experiences with Ms. Inquisitive. With the help of fictional characters, the paper scrutinizes and suggests ways to construct, contextualize, administer and analyse Vignettes.
8. Implementing the Most Significant Change technique for monitoring and evaluation of social and behavior change programme: process, outcomes and lessons learnt
The ‘Most Significant Change’ technique is increasingly gaining popularity among project/programme managers as a qualitative Participatory Monitoring and Evaluation method. The technique involves collection of ‘Significant Change’ stories of the clients in a programme/project and then selection of changes considered as Most Significant by the clients in a multistage process. With focus on changes considered as significant by the clients, Most Significant Change Technique gets distinguished from other Monitoring and Evaluation Techniques as it enables assessment of indirect outcomes of a programme/project that usually are not captured by conventional indicator-based Monitoring and Evaluation systems or other techniques of Participatory Assessment, Monitoring and Evaluation.
Since 2008, UNICEF India has been implementing the Most Significant Change technique in its Social and Behavior Change Programmes in India. With “Communication for Development” perspective, the programmes looks at Behavior Change Communication as a vehicle for Social Change. MSC technique has been employed in the programmes as a qualitative participatory monitoring and evaluation technique alongside the conventional indicator based monitoring systems. The technique is employed to provide insights into the direct and indirect outcomes incidental to programme implementation.
This paper is based on experiences in implementation of the technique. It discusses the implementation processes adopted, the analysis paradigm and outcomes being reflected by analyses of the Most Significant Change stories. It also delineates lessons learnt from the implementation experience that provide a feed-forward for large-scale implementation of the technique
9. Scaling up social innovations: The 4D model
Demand responsive models or community engaging social ideas had always been the salt for the development sector. However, the efforts had mostly been localized and transitory. Often the valuable innovations are lost as just a drop of excellence rather than a model of replication. Though a sea of literature is available on strategies and factors that enable scale up of an idea, scaling a community focused innovation has not been much researched about.
Often the idea is supposed to exist in a certain universe; the real opportunity is when it is adopted by a larger universe as a process of social change. Scale–up thus turns into a multiplicative strategy for making an impact felt to a greater population.
The paper attempts to devise a strategic framework for taking a community focused innovation from pilot to scale. It looks into how the designs, underlying purposes, changing political and social contexts and identities foment the uptake and scale up of an innovation. The paper analyses the variables to base assumptions on and the way the assumptions drive the acceptance of an innovation.
13. Using The Kirkpatrick Model Of Training Evaluation
Investments in capacity building are a key feature of development interventions underscoring the role of capacity development in enhanced developmental outcomes and sustained change. Also, trainings have become the mainstay of these interventions such that to be synonymous to capacity building. And therefore evaluation of trainings to comment on efficacy of the interventions, efficiency of the investments and contribution to envisioned results becomes imperative. However, the practice of training evaluation is limited with methodological challenges, skills, resource and time constraint being the key contributors.
This paper provides and overview of the various approaches to training and discusses the four-level Kirkpatrick model of training evaluation in detail. It describes the use of the model in training evaluation of a health sector intervention. It describes the methodology, tools and analyses protocols employed for the evaluation and highlight the challenges in application of the model to comprehensively evaluate training interventions.
14. Quantitative Approach To Forecasting Using “R”
Forecasting has been playing a major role in various spheres of life and has assumed significant importance during recent years. It has found its applications in varied domains such as Economic Forecasting, Technology Forecasting, Weather Forecasting, Sales Forecasting, Demand Forecasting etc.
The paper attempts to understand various Quantitative Forecasting Techniques and explores reasons for using a particular method in a given study scenario. The paper also describes the applications of different methods and lists down the processes of applying those techniques using Objective Approach of Time Series Models through “R Statistics Software”. Though the paper tries to answer such questions on theoretical grounds, it also attempts at providing practitioner’s point of view so as to enable applications of Time Series Forecasting Techniques using Exponential Smoothing Methods in practice.
15. Exploring Standardized Patient Technique For Monitoring Quality Of Care In Health Service Delivery
The ever emerging thrust for quality improvement in health care practices often look forward to appropriate assessment scholarship. The need for newer approaches for assessment and continuation of older methods generally come into conflict concerning the uncertainties in quality improvement. Assessing quality care as well as gaps in knowledge and practice of healthcare providers, Standardized Patient technique is currently being used in developing countries. The technique employs well-trained field workers to take on the role of patients and visit a doctor just like an actual patient would. The Standardized Patient is expected to pay attention during the interaction and record all observations in form of notes or in an exit-tool; the central idea being simulation of an actual patient environment. Through standardization and simulation, it tries to analyse process quality and determinants of quality. Standardized Patient as a tool is compared with medical vignette and direct clinical observation. It has been argued that Standardized Patient could work as an efficient tool in minimizing the Hawthorne effect.
The purpose of the paper is to discuss Standardized Patient as an evolving health care monitoring tool. The paper thus focuses on the relevance of technique and ascertain current usage in monitoring health systems through secondary data and review of literature to enumerate the best practices. Drawing upon examples from projects implemented or currently underway, the paper also explores modalities for implementation of the technique in an Indian setting.