The World Bank

Evaluation Summary What Can We Learn from Nutrition Impact Evaluations? High levels of child malnutrition in developing countries contribute to mortality and have long-term consequences for children s cognitive development and earnings as adults. Recent impact evaluations show that many different interventions have had an impact on children s anthropometric outcomes (height, weight, and birth weight), but there is no simple answer to the question What works? to address the problem. Similar interventions have widely different results in different settings, owing to differences in local context, the causes and severity of malnutrition, and the capacity for program implementation. Impact evaluations of programs supported by the Bank, which are generally large-scale, complex inter-ventions in low-capacity settings, show equally variable results. The findings confirm that it should not be assumed that an intervention found effective in a randomized medical setting will have the same effects when implemented under field conditions. There are many robust experimental and quasi-experimental methods for assessing impact under difficult circumstances often found in field settings. The relevance and impact of nutrition impact evaluations could be enhanced by collecting data on service delivery, demand-side behavioral outcomes, and implementation processes to better understand the causal chain and what part of the chain is weak, in parallel with impact evaluations. It is also important to understand better the distribution of impacts, particularly among the poor, and to document better the costs and effectiveness of interventions. High levels of child malnutrition in developing countries are contributing to mortality and present long-term consequences for the survivors. An estimated 178 million children under age 5 in developing countries are stunted (low height for age) and 55 million are wasted (low weight for height). Malnutrition makes children more susceptible to illness and strongly affects child mortality. Beyond the mortality risk in the short run, the developmental delays caused by undernutrition affect children s cognitive outcomes and productive potential as adults. Micronutrient deficiencies vitamin A, iron, zinc, iodine, for example are also common and have significant consequences. Progress in reducing malnutrition has been slow: More than half of countries are not on track to achieve the Millennium Development Goal of halving the share of children who are malnou-rished (low weight for age) by 2015. The food price and financial crises are making achievement of this goal even more elusive. The World Bank has recently taken steps to ex-pand its support for nutrition in response to the underlying need and the increased urgency due to the crises. WHAT DO WE KNOW ABOUT REDUCING MALNUTRITION? The increased interest and resources focused on the problem of high and potentially increasing rates of undernutrition raises the question, what do we know about the causes of malnutrition and the in-terventions most likely to reduce it? The medical literature points to the need to inter-vene during gestation and the first two years of life to prevent child malnutrition and its consequences. It suggests that investments in interventions during this window of opportunity among children under 2 are likely to have the greatest benefits. Recently published meta-analyses of the impact evaluation literature point to several interventions found effective for reducing undernutrition in spe-cific settings. However, there are limitations to the generalizability of those reviews findings, particularly in the context of large-scale government programs most likely to be supported by the World Bank. The reviews tend to disproportionately draw on the findings of smaller, controlled experiments; there are few examples of evaluations of large-scale programs, over which there is less control in implementation. In reviewing a large number of studies, interventions, and outcomes, they tend to focus on average impacts. They generally do not explain the magnitude or variability of impacts across or within studies. Very few address the programmatic reasons why some interventions work or don t work, nor do they assess the cost-effectiveness of interventions. Objectives of the Review This paper reviews recent impact evaluations of interventions and programs to improve child anth-ropometric outcomes height, weight, and birth weight with an emphasis on both the findings and limitations of the literature and on understanding what might happen in a non-research setting. It further reviews in greater detail the experience and lessons from evaluations of the impact of World Bank-supported programs on nutrition outcomes. Specifically, the review addresses four questions. First, what can be said about the impact of different interventions on children s anthropometric outcomes? Second, how do these findings vary across settings and within target groups, and what accounts for this variability? Third, what is the evidence of the cost-effectiveness of these interventions? Finally, what have been the lessons from implementing impact evaluations of Bank-supported programs with anthropometric impacts? While there are different dimensions of child nutri-tion that could be explored, the report focuses on child anthropometric outcomes -- weight, height, and birth weight. These are the most common nutrition outcome indicators in the literature and the most frequently monitored by national nutrition programs supported by the World Bank. Low weight for age (underweight) is also the indicator for one of the MDGs. Methodology and Scope Forty-six nutrition impact evaluations published since 2000 were systematically reviewed. These evaluations assessed the impact of diverse interven-tions community nutrition programs, conditional and unconditional cash transfers, early child devel-opment programs, food aid, integrated health and nutrition services, and de-worming. All of the evaluations used research designs that compared the outcomes among those affected by the project to the counterfactual that is, what would have happened to a similar group of people in the absence of the intervention. About half used randomized assignment to create treatment and control groups, while the remainder used matching and various econometric techniques to construct a counterfactual. Among the 46 evaluations, twelve assessed the im-pact of World Bank-supported programs on nutri-tion outcomes in eight countries. While the broader review relies on the analysis of the published impact evaluations as the main source of data, for these twelve evaluations project documents and research outputs were reviewed and World Bank staff, country officials and the evaluators and re-searchers who conducted the studies were interviewed. Findings A wide range of interventions had a positive impact on indicators related to height, weight, wasting, and low birth weight. There were a total of 10 different outcome indica-tors for the four main anthropometric outcomes. A little more than half of the evaluations addressing a height-related indicator found program impacts on at least one group of children, and this was true for about the same share of interventions aimed at improving weight-related and wasting (low weight for height)-related indicators. About three-quarters of the 11 evaluations of interventions that aimed at improving birth weight indicators registered an impact in at least one specification, including five out of seven micronutrient interven-tions. There was no clear pattern of impacts across interventions in every intervention group there were examples of programs that did and did not have an impact on a given indicator, and with varying magnitude. Evaluations of the nutritional impact of programs supported by the World Bank, which are generally large-scale, complex, and implemented in low-capacity settings, show equally variable results. Even controlling for the specific outcome indicator, studies often targeted children of different age groups that might be more or less susceptible to the interventions. It is thus difficult to point to inter-ventions that are systematically more effective than others in reducing malnutrition across diverse set-tings and age groups. Differences in local context, variation in the age of the children studied, the length of exposure to the intervention, and differing methodologies of the studies account for much of the variability in results. Context includes factors like the level and local determinants of malnutrition, differences in the characteristics of beneficiaries (including their age), the availability of service infrastructure, and the implementation capacity of government. Outside of a research setting in the context of a large government program there are many things that can go wrong in either service delivery or the demand response that can compromise impact. Beyond this, there are social factors like the status of women or the presence of civil unrest that can affect outcomes. These findings underscore the conclusion that it should not be assumed that an intervention found effective in a randomized controlled trial in a re-search setting will have the same effects when im-plemented under field conditions in a different set-ting. They also point to the need to understand the prevailing underlying causes of malnutrition in a given setting and the age groups most likely to benefit in selecting an intervention. Further, impact evaluations need to supplement data measuring impact with data on service delivery and demand-side behavioral outcomes to demonstrate the plausibility of the findings, to understand what part of a program works, and to address weak links in the results chain to improve performance. There is scant evidence on the distribution of nutrition impacts who is benefiting and who is not or on the cost-effectiveness of interventions Just because malnutrition is more common among the poor does not mean that they will disproportio-nately benefit from an intervention, particularly if acting on new knowledge or different incentives relies on access to education or quality services. Only a third of the 46 evaluations looked at the distribution of impacts by gender, mother s education, poverty status, or availability of complementary health services. Only nine assessed the impacts on nutritional outcomes of the poor compared with the non-poor. Among the evaluations that did examine variation in results, several found that the children of more educated mothers or in better-off communities are be-nefitting the most. Bank-supported cash transfers, community nutrition, and early child development programs in six of eight countries had some impact on child anthropometric outcomes. Of the 12 impact evaluations of Bank support, all but one were of large-scale government programs with multiple interventions and a long results chain. Three-quarters found a positive impact on anthro-pometric outcomes of children in at least one age group, although the magnitude was in some cases not large or applied to a narrow age group. Most of the impact evaluations involved assessment of completely new programs and involved World Bank researchers. Most used quasi-experimental evaluation designs and two-thirds assessed impact after at most 3 years of program implementation. Only half of the evaluations documented the distribution of impacts and only a third presented information on the costs of the intervention (falling short of cost-effectiveness analysis). In two of the countries (Colombia and the Philippines) the evaluations likely had an impact on government policy or programs. Lessons A number of lessons for development practi-tioners and evaluators arose from the review of impact evaluations of World Bank nutrition support. For task managers: Impact evaluations of interventions that are clearly beyond the means of the government to sustain are of limited relevance. The complexity, costs, and fiscal sustainability of the intervention should figure into the decision as to whether an impact evaluation is warranted. Impact evaluations are often launched for the purpose of evaluating completely new pro-grams, but they may be equally or even more useful in improving the effectiveness of ongo-ing programs. There are methods for obtaining reliable impact evaluation results when randomized assignment of interventions is not possible for political, ethical, or practical reasons. For evaluators: In light of the challenges of evaluating large-scale programs with a long results chain, it is well worth the effort to assess the risks to disruption of the impact evaluation ahead of time and identify mitigation measures. The design and analysis of nutrition impact evaluations need to take into account the likely sensitivity of children of different ages to the intervention. For the purposes of correctly gauging im-pact, it is important to know exactly when delivery of an intervention took place in the field (as opposed to the official start of the program). Evaluations need to be designed to provide evidence for timely decision-making, but with sufficient elapsed time for a plausible impact to have occurred. The relevance of impact evaluations for po-licymakers would be greatly enhanced if im-pact evaluations were to document both the

Lessons from a Review of Interventions to Reduce Child Malnutrition in Developing Countries

What Can We Learn from Nutrition Impact Evaluations?

OECD

Cette publication examine le concept de renforcement des capacités par l'enseignement supérieur transnational et met l'accent sur le rôle clé de l'assurance qualité et des négociations commerciales.

Un levier pour le développement

L'enseignement supérieur transnational Un levier pour le développement

This book discusses the concept of capacity-building for tertiary education through cross-border education, emphasising the critical role of quality assurance and trade negotiations.

A Way towards Capacity Development

Cross-border Tertiary Education A Way towards Capacity Development

Food and Agriculture Organization of the United Nations 

This Inter-Agency Social Protection Assessment (ISPA) tool on Food Security and Nutrition (FSN ISPA) provides a framework of analysis to assess how social assistance and programmes can achieve a greater positive impact on food security and nutrition (FSN) at country level. This tool belongs to the set of programme level ISPA tools and focuses on identifying opportunities to strengthen coherence and improve the results of social assistance programmes for FSN. The FSN ISPA tool is a broadly designed proposed set of principles, steps and instruments for country adaptation that must be tailored to each country context and programme under review. It is not intended to be used for cross-country comparisons, but rather aims to analyse the given programme(s) of focus and country context (including FSN and other conditions), to identify where and how FSN impacts may be enhanced. ISPA tools are the result of a multi-agency initiative that aims to put forth a unified set of definitions, assessment tools, and outcome metrics to provide systematic information for a country to assess its social protection system, schemes, programmes, and implementation arrangements.

Inter Agency Social Protection Assessment (ISPA) tool on Food Security and Nutrition

Inequality in South Asia appears to be moderate when looking at standard indicators such as the Gini index, which are based on consumption expenditures per capita. But other pieces of evidence reveal enormous gaps, from extravagant wealth at one end to lack of access to the most basic services at the other. Which prompts the question: How bad is inequality in South Asia? And why would that matter? This book takes a comprehensive look at the extent, nature, and drivers of inequality in this very dynamic region of the world. It discusses how some dimensions of inequality, such as high returns to investments in human capital, contribute to economic growth while others, such as high payoffs to rent-seeking or broken aspirations, undermine it. Drawing upon a variety of data sources, it disentangles the contribution that opportunity in young age, mobility in adult years, and support throughout life make to inequality at any point in time. Equally important, the book sheds light on the prospects of escaping disadvantage over time. The analysis shows that South Asia performs poorly in terms of opportunity. Access to basic services is partial at best, and can be traced to characteristics at birth, including gender, location, and caste. Conversely, the region has had a robust performance in terms of geographical and occupational mobility despite its cluttered urbanization and widespread informality. Migration and jobs have served disadvantaged groups better than the rest, highlighting the importance of the urbanization and private sector development agendas. Support falls somewhere in between. Poverty alleviation programs are pervasive. But the mobilization of public resources is limited and much of it is wasted in regressive subsidies, while inter-government transfers do not do enough to mitigate spatial inequalities.

Addressing Inequality in South Asia

The evaluation finds that the content of the World Bank s Country Policy and Institutional Assessment (CPIA) is largely relevant for growth and poverty reduction in the sense that it maps well with the determinants of growth and poverty reduction identified in the economics literature. However, some CPIA criteria need to be revised (in particular trade and finance), and one needs to be added (assessment of disadvantaged socio-economic groups). Second, the evaluation finds that the CPIA ratings are in general reliable and correlate well with similar indicators. The World Bank s internal review process helps guard against potential biases in having Bank staff rate countries on which their work programs depend. The CPIA ratings are found to correlate better with similar indicators for middle income countries than for low income countries. This could be because there is more information available on middle income countries, which increases the likelihood of different institutions having similar assessments on them. This could also be because the CPIA rating exercise takes into account the stage of development, which is more pertinent for low income countries, and which also subject the ratings of those countries to more judgment in an exercise that is already centered on staff judgment.

An IEG Evaluation

The World Bank's Country Policy and Institutional Assessment

This book explores emerging topics in innovation policy for more inclusive and sustainable growth, building on concrete examples, and develops the notion of experimental innovation policy.

Learning from Experimentation

Making Innovation Policy Work Learning from Experimentation

Guangdong, a province of over 93 million residents, is located on the southern coast of China, boarding with Hong Kong, China. As China s powerhouse for economic growth and a pioneer of reform and opening up, Guangdong has maintained an annual average GDP growth rate of 13.7 percent over the past three decades. Its historical achievements notwithstanding, Guangdong witnessed increased inequality and regional disparity. To assist the authority in developing a strategy for the new phase of reforms that promotes more inclusive and sustainable growth, Reducing Inequality for Shared Growth in Guangdong Province recommends a three-pillar approach: eliminating absolute poverty, reducing inequality in opportunities, and containing inequality in outcomes. The book also proposes a range of policy actions in these three broad areas. First, to further develop the social assistance program (i.e. the minimum living allowance program ) to address the issue of absolute poverty; Second, to improve income opportunities of the rural poor by better facilitating rural labor migration to non-farming jobs and urban labor markets, deepening rural finance reform, and providing better protection of their rights over land. Third, to invest in people through more equitable access to and financing of social services such as basic education, skills development, and health care. Further reform of the intergovernmental fiscal system is essential to the success of these efforts. This report will be of interest to central and sub-national policy makers, policy implementing agencies, researchers, development partners, and others working on economic and social development in China and in other countries. Guangdong s experience will offer great value to the rest of China and to other countries that are grappling with similar development challenges.

Strategy and Policy Options for Guangdong Province

Reducing Inequality for Shared Growth in China

The rise in global food prices and the ever-growing food import bill have prompted sharp attention on agricultural policies in Africa. African policy makers are grappling with what unstable food prices mean for their countries; how these price movements will affect their food security situation; how the private sector is likely to respond; and what governments themselves can do. In addition, they fear that global warming may significantly change the location of food production within Africa. This report discusses how opening up cross-border trade will boost the potential for greater food production in Africa and contribute to food security by improving poor people’s access to food and by increasing returns to poor farmers for the food they produce.

Africa Can Help Feed Africa

Samer Al-Samarrai

Blane Lewis

The majority of the world’s children live in countries where local governments are responsible for the provision of basic education services. Although subnational governments manage their own education systems, they often rely on transfers from the central government for funding. The main purpose of this study is to assess how these fiscal transfers affect public funding for education and how they ultimately affect student schooling and learning outcomes. Through a careful analysis of how fiscal transfers have affected education systems in different contexts, the investigation develops a set of principles to support improvements in the design and implementation of transfer systems with a specific focus on the provision of education services. The study is centered on seven country case studies that aim to answer a set of common research questions using a similar approach. Country case studies were conducted in Brazil, Bulgaria, China, Colombia, Indonesia, Sudan, and Uganda. The analysis shows that fiscal transfer mechanisms can improve the adequacy of public education spending, reduce spending inequalities between regions, and improve spending efficiency. Moreover, the study highlights that carefully designed and implemented transfer systems can help raise overall education outcomes and reduce education inequality. This publication was funded by a grant from the Results in Education for All Children (REACH) trust fund at the World Bank. REACH is supported by the government of Germany through theFederal Ministry for Economic Cooperation and Development, the government of Norway through NORAD, and the government of the United States of America through the U.S. Agency for International Development.

The Role of Intergovernmental Fiscal Transfers in Improving Education Outcomes

International Monetary Fund

International Labour Office

The United Nations Economic Commission for Europe

Statistical Office of the European Communities, Luxembourg

The consumer price index (CPI) measures the rate at which prices of consumer goods and services change over time. It is used as a key indicator of economic performance, as well as in the setting of monetary and socio-economic policy such as indexation of wages and social security benefits, purchasing power parities and inflation measures. This manual contains methodological guidelines for statistical offices and other agencies responsible for constructing and calculating CPIs, and also examines underlying economic and statistical concepts involved. Topics covered include: expenditure weights, sampling, price collection, quality adjustment, sampling, price indices calculations, errors and bias, organisation and management, dissemination, index number theory, durables and user costs.

Theory and Practice

Consumer Price Index Manual

This joint OECD-World Bank stocktaking exercise of social accountability (SA) initiatives in OECD member countries contributes to the global exchange of policy knowledge. The stocktaking exercise produced 40 templates detailing social accountability initiatives in 27 OECD countries and the EC.

Stocktaking of Social Accountability in OECD Countries

Beyond Public Scrutiny Stocktaking of Social Accountability in OECD Countries

United Nations Environment Programme

This report is a joint effort by the OECD, UN Environment and the World Bank Group, supported by the German Federal Ministry for the Environment, Nature Conservation and Nuclear Safety. It focuses on how governments can move beyond the current incremental approach to climate action.

Rethinking Infrastructure

Financing Climate Futures Rethinking Infrastructure

The report proposes a four-pronged framework which will empower Korea to effectively tap into technology advances and the growing stock of knowledge.

Making the Transition

Korea and the Knowledge-based Economy Making the Transition

The World Development Report 2022 is a practical guide to help low-and-middle-income countries strengthen financial markets for a more equitable recovery from the COVID-19 economic crisis.

Finance for an Equitable Recovery

World Development Report 2022

The World Bank Legal Review, Volume 2: Law, Equity and Development

The report analyzes key challenges for improving gender equality in the MENA region and provides policy priorities that Governments could consider to address these challenges. By and large the critical areas are in improving economic and political participation of females.

Gender Equality and Development in the Middle East and North Africa

Opening Doors

Miria Pigato

Technology Transfer and Innovation for Low-Carbon Development

Development Research Center of the State Council

In the last 30 years, China’s record economic growth lifted half a billion people out of poverty, with rapid urbanization providing abundant labor, cheap land, and good infrastructure. While China has avoided some of the common ills of urbanization, strains are showing as inefficient land development leads to urban sprawl and ghost towns, pollution threatens people’s health, and farmland and water resources are becoming scarce. With China’s urban population projected to rise to about one billion – or close to 70 percent of the country’s population – by 2030, China’s leaders are seeking a more coordinated urbanization process. Urban China is a joint research report by a team from the World Bank and the Development Research Center of China’s State Council which was established to address the challenges and opportunities of urbanization in China and to help China forge a new model of urbanization. The report takes as its point of departure the conviction that China's urbanization can become more efficient, inclusive, and sustainable. However, it stresses that achieving this vision will require strong support from both government and the markets for policy reforms in a number of area. The report proposes six main areas for reform: first, amending land management institutions to foster more efficient land use, denser cities, modernized agriculture, and more equitable wealth distribution; second, adjusting the hukou household registration system to increase labor mobility and provide urban migrant workers equal access to a common standard of public services; third, placing urban finances on a more sustainable footing while fostering financial discipline among local governments; fourth, improving urban planning to enhance connectivity and encourage scale and agglomeration economies; fifth, reducing environmental pressures through more efficient resource management; and sixth, improving governance at the local level.

Toward Efficient, Inclusive, and Sustainable Urbanization

Urban China

This second edition of Integrating Human Rights into Development consolidates the findings and research compiled in 2006 with key developments and activities that have occurred in the intervening seven years.

The World Bank

Books

Your next favorite book is out there