The Impact Evaluation Problem Facing the Development Sector

humanitarian aid development sector

The development sector faces a growing impact evaluation problem. While not new, the issue of transparency and accountability is becoming an increasingly pressing one for the sector to confront.

While you’re here, check out Why AOC Would Have Wiped the Floor with Ben Shapiro

How Big Is The Aid Sector?

According to one study out of Johns Hopkins University, were the development sector a country it would have the world’s 5th largest GDP. Other figures float around: American individuals, corporations, and private philanthropic organizations spent $390 billion a year giving to charities and NGOs; Development Assistance Committee (DAC) member countries and EU institutions hand over around $160 billion annually in Official Development Assistance (ODA); formal non-DAC assistance – led by China – amounts to around $60 billion a year.

Hard figures are difficult to come by. But needless to say, the aid sector is substantial. But does all that money have a positive impact on the communities development actors serve? And is that impact long-term? How is it measured?

In A World Of Big Data, The Development Sector Seems To Lack Data

Impact evaluation data is not easy to gather in the development sector, and much of that is due to the challenging environments in which a lot of aid is provided. The relative scarcity of reliable impact data is despite the fact that most participants tend, and are often mandated, to spend between 5 to 8 percent of their budgets measuring their impact.

What is increasingly becoming recognized as essential is long-term impact assessment: the best evidence that development projects are successful and achieving their goals is that they are having a long-term impact.

A significant area of impact assessment weakness for development actors is their lack of access to, and minimal capacity to obtain, ‘tracer data’, that is – data that follows aid beneficiaries’ trajectories over longer-term time horizons to determine the long-term impact of interventions. Tracer data is now becoming a focal point for the industry, which now realizes that short-term impact evaluations may be painting an incomplete picture.

According to the International Labour Organization (ILO), a tracer study is a study that “explores what changes occurred in the lives of former beneficiaries, and if and how the intervention contributed to these changes”.

The organization goes on to say:

“The principal goals of tracer studies are to document changes in the lives of former beneficiaries, find out what these former participants are currently doing and gain insight that will allow organizations to improve their performance and better plan projects and activities… Implementing agencies usually have a fairly good idea of what the [beneficiaries] are doing for the duration of the project, and at the moment it ends. However, the real challenge is for interventions to produce lasting, long-term results. The ultimate measure of the success of an intervention is for the desired changes in the lives of [beneficiaries] to still be present, in some manner, after the project itself has phased out.”

Tracer data is currently prohibitively expensive to obtain unless data systems are inaugurated during the implementation phase of a program. In an era where data and the tools to collect it are becoming ubiquitous, let’s hope the benefits spread to the developing world.