The Imperative of Evidence-Based Development Cooperation

In a world facing multiple intersectional crises, from the COVID-19 pandemic to impending climate change, government development agencies face a monumental task when prioritizing international assistance. Often, these choices are heavily influenced by political considerations, domestic interests and foreign policy objectives, at times forgoing proven effective programs supported by robust evidence.

For instance, many governments prioritize addressing the roots causes of irregular migration through development cooperation, despite mixed or inconclusive evidence regarding effective irregular migration interventions. While solely evidence-based development cooperation may be out of reach, integrating evidence-based decision-making and robust evidence into policy planning and program management is a feasible political decision that complements many other considerations.

Another instance of evidence-based decision-making is the ability to evaluate cost-effective health interventions. While some interventions are very cost-effective, requiring less than US$200 per disability-adjusted life years averted, others cost up to US$1000. Understanding the cost-effectiveness of different health interventions can help a government with limited resources decide where to allocate its global health budget. For example, preventive chemotherapy for onchocerciasis, a parasitic worm infection, is a highly cost-effective intervention in low-income countries, whereas other health interventions are much less cost-effective in this context. While factors such as equity, affordability, and sustainability are highly important, robust evidence-based information provides valuable guidance for effectively prioritizing global health spending.

This Donor Tracker Partner Perspective, in collaboration with Coopération Globale, discusses the importance of evidence-based decision-making in development cooperation. It highlights approaches and practices to promote the use of robust evidence, emphasizing the need for governments to incorporate robust methodologies to maximize the impact of their development cooperation efforts and generate significant change around the world.

Defining Robust Evidence

There is a vast advantage to using evidence based on scientific evaluations, also known as robust evidence, in comparison to traditional evidence, which is based solely on evaluating outcomes without investigating deeper causes. Robust evidence uncovers and illuminates in much greater detail than traditional evaluation methods.

For instance, observing that children’s literacy has improved following a school intervention such as improving a curriculum, as might be done with traditional evidence-gathering, cannot conclusively attribute the improvement to that specific intervention. It could be the result of unforeseen or unintended factors, such as increased parental involvement in learning at home, that was not part of the intervention. Robust evidence, in contrast, would be able to isolate the intervention's effect by comparing a treatment group to a comparative control group, isolating the true cause behind the improvements.

Another example of robust evidence generation is a methodology known as cash benchmarking, which compares the outcomes of a project or program with a scenario where beneficiaries receive cash transfers of equal value to the cost of implementation. This approach recognizes that recipients often have a better understanding of how to use funds to improve their lives than program designers, and are best able to accommodate their own unique needs and contexts. If the outcomes achieved by the programs do not surpass the benefits of cash transfers, it would be prudent to reconsider the program design prior to approval.

Best Practices

Several development actors have already recognized the importance of evidence-based approaches and have taken steps and initiatives to implement it in their development cooperation initiatives.

US / USAID

In 2011, USAID adopted its current Evaluation Policy. This policy champions a robust evaluation practice rooted in sound planning and design, independent judgment, high-quality methodologies, and evidence-based findings. Since then, USAID's organizational culture has experienced a significant shift, moving beyond a compliance mindset to a more holistic evaluation of development cooperation.

For example, during 2011-2016 these changes included:

• Leadership Commitment: Agency leaders emphasized the importance of evidence in decision-making, fostering a culture of substantiated answers informed by evaluation;
• Incentivizing Quality: Conducting excellence in evaluation contests and independent studies to improve evaluation excellence;
• Building Staff Capacity: Offering formal training, workshops, technical assistance in monitoring & evaluation, and recruiting for specialized staff; and
• Networking: Establishing internal communities of practice and collaborating externally to stay updated on evaluation methods.

These efforts continued with USAID’s Agency Learning Agenda 2022-2026, which defined learning questions and activities geared towards improving the generation, synthesis, sharing, and use of evidence.

USAID recently hired Dean Karlan, the founder of Innovations for Poverty Action and a leading development economist, in its pursuit of channeling much of its $41 billion annual budget into proven, cost-effective evidence-based programs.

Norway / Norad

Norway recently articulated its commitment to maintain high quality and knowledge-based development cooperation. This declaration underscored the integration of impact evaluation and formative research into development programs.

In line with this overarching commitment, Norad launched the Impact Evaluation Incubator, which is a transformative three-day workshop funded by Norad for NGOs, equipping them with the essential skills to embrace best practices in evaluation methodology and effectively address practical challenges.

Similarly, Norad approved a policy requiring cost analysis in all impact evaluations, allowing for effectiveness analysis and providing better, more relevant information for future policy decisions and policymaking.

UK / FCDO

The UK's FCDO introduced an initiative known as Knowledge, Evidence and Learning for Development. The initiative is a research helpdesk for FCDO as well as other UK government departments and research partners, providing rapid access to evidence and analytical insights pertaining to development policy and programs. The helpdesk encompasses both standard reports and more in-depth analyses addressing emerging issues and learning products. Furthermore, it serves as a platform for fostering dialogue and reflection linked to real-world case scenarios and practical application.

Given its success, a follow-up initiative is in the works, which is slated to build on the lessons learned of the first phase and tackle further, non - ODA-related issues in a broader set of tasks involving development and diplomacy.

Elevating Robust Evidence and Room for Improvement

Governments are at different stages in the process of recognizing and harnessing robust evidence in public policies. Switzerland and Germany, for example, have made some limited progress incorporating robust evidence, but there is still great potential.

The OECD accorded Switzerland a good overview of its development cooperation, and its evaluation performance is also generally positively assessed. In 2008, the Swiss Agency for Development and Cooperation decided to spend 0.6-0.8% of its total budget on external and internal evaluations each year. Accordingly, it commissions numerous thematic and institutional evaluations each year, which are carried out by independent and external teams of experts who organize evaluations of country and regional cooperation programs both domestically and abroad.

However, rigorous impact evaluations constitute only about 2% of all evaluation and research projects and constitute an extremely minor fraction of evaluations(according to public database and based on own calculations). This observation becomes particularly significant contrasted with the fact that a 10% share of similar rigorous evaluations at USAID is still perceived as relatively low. Switzerland’s ongoing development of its 2025-2028 international cooperation strategy, alongside its current examination of project evaluation quality, presents an opportunity to firmly anchor robust evidence in future development cooperation practices.

Germany has begun to welcome initiatives to further evidence-based approaches, including the Rigorous Impact Evaluation initiative. This initiative includes a robust evidence database comprising, amongst other features, numerous rigorous impact evaluations conducted by the GIZ. Germany has also launched the KfW Interactive Database for Evaluation and Learning, which provides easily accessible, filterable, and individually tailored evidence and knowledge for projects and programs under the BMZ. However, an OECD report underscored a recommendation for Germany to enhance and create a more results-oriented management and results-driven culture, capable of assessing and evaluating its thematic, regional, and global programs and learning. A study on the status quo of rigorous impact evaluations revealed that the uptake of rigorous evidence in German development cooperation is low.

A promising next step to strengthen robust evidence usage and generation includes setting up evidence hotlines to support program managers and making references to robust evidence mandatory in project proposals, amplifying the role and functionality of the German Institute for Development Evaluation to provide robust evidence, pertinent at both the strategic program/, project, and policy levels.

Key Recommendations

Based on examples from leading development actors, several key informant interviews with development agency officials and consultants, as well as academic research, the following actions are recommended for governments to implement robust evidence in development cooperation and policymaking:

1. Strengthening the Use of Robust Evidence:

• Incorporate evidence in project cycle management: Mandate the use of existing evidence in proposals for project and program managers, as well as stakeholders;
• Enhance Evidence Accessibility: Create user-friendly databases for project proposals, monitoring & evaluation data, and evaluations, including third-party research (e.g., the World Bank’s 2023 Cost-effective Approaches to Improve Global Learning) and recommendations from third-party platforms (e.g., 3ie, J-PAL, IPA);
• Quality Criteria: Evaluate existing evidence against defined quality criteria, considering methodological rigor and contextual relevance; and
• Policy-Relevant Evaluations: Combine robust evaluations with other analyses, such as cost analyses, for greater policy relevance.

2. Strengthening Robust Evidence Generation

• Minimum Quota of Rigorous Impact Evaluations: Establish a minimum number of high-quality rigorous impact evaluations as a percentage of operational expenditures in all projects above a certain budget threshold and in all scalable pilot projects;
• High-Quality Evaluation Guidelines and Standards: Harmonize evaluation guidelines and criteria across field offices and partners; and
• Support for Rigorous Interventions: Contribute to institutions scaling rigorously tested interventions (e.g., Global Innovation Fund, fid).

3. Fostering an Evidence-Based Culture

• Capacity building: Enhance the knowledge of and capacity to generate robust evidence of employees engaged in monitoring & evaluation, project & program planning, and policymaking;
• Support for Staff: Provide evaluation helpdesks tailored to specific bottlenecks and questions related to evidence and rigorous impact evaluations (e.g., where to find relevant evidence, what evidence to use, how to design and organize an impact evaluation);
• Knowledge Retention: Implement non-rotational policies for designated roles within thematic and evaluation departments to incentivize and leverage professional expertise in robust evidence;
• Independent Evaluations: Establish an evaluation unit with institutional independence from project & program implementation units; and
• Leadership Commitment: Ensure leadership commitment to prioritizing evidence in development cooperation and beyond.

Conclusion

Evidence-based development cooperation is crucial for governments to save lives, enhance development cooperation, and ensure maximum effectiveness of taxpayer-funded programs. Policies supported by robust evidence can ensure that resources are directed to where they generate the most significant impact.

While challenges to implementation exist, many governmental development cooperation agencies have already successfully launched initiatives to integrate robust evidence into decision-making. Implementing the recommended actions listed above can lead to great advancements in the development cooperation landscape and tangible benefits for those who need it most.