blog logo image

Why Management Reform Is Sexy

July 16th, 2014
Bookmark and Share

See below for a guest post by Andrea Koppel, Vice President of Global Engagement and Policy at Mercy Corps and MFAN Executive Committee member.

***

I have a confession to make.   I may have a serious day job fighting for better policies to help vulnerable people around the world, but after hours I’m an avid consumer of tabloids.   Grocery store check-out lines – the longer the better.  Hair salons – I don’t mind waiting. Airport kiosks are my preferred venue for binge reading.  I’d choose a salacious, rumor-mongering US magazine over the more serious, well-sourced Foreign Affairs any day of the week.

Even at the uber intellectual State Department it’s hard to compete against sexy. Take for example ongoing discussions around selecting overarching foreign policy and development objectives for the next four years – a document known as the Quadrennial Diplomacy and Development Review (QDDR II).

Good luck making that short list.  With titillating topics such as climate change, which Secretary of State John Kerry calls “as big a threat as terrorism”, and exciting new ventures such as USAID’s Global Development Lab, “management reform” risks being ignored like a wonky wallflower.

But just as a wealthy older man believes he’ll attract a trophy wife, I’d like to believe that much needed “management reform” will still catch the eye of QDDR II’s head honcho Tom Perriello.  If you read beyond the headline to see just how valuable these reforms can be – both in terms of efficiencies and impact – there’s a lot to like.

Let’s face it – while global poverty has decreased over the last 25 years, the number of extremely poor people has stayed about the same in the most fragile and conflict-affected countries.   By 2018, roughly half of the world’s extreme poor will live in fragile states.  Many of these countries are of strategic importance to the U.S., yet they are also places where U.S. or international assistance has had little success in breaking cycles of conflict or poverty, despite two decades of concerted international development and humanitarian responses.   To try to figure out why that’s been the case, the Mercy Corps Policy and Advocacy team interviewed more than two dozen of our most experienced colleagues – many of whom live and work in these tough places.

Our conclusion is that if the Obama administration is to truly build momentum for its ambitious policy directive to eradicate extreme poverty in the world by 2030, it must address five distinct, yet interconnected problems:

1. U.S. conflict mitigation and management structures are disproportionately oriented towards crisis response, rather than crisis prevention or resilience building.

2. Few funding mechanisms address the underlying causes of extreme poverty or chronic insecurity.

3. Foreign assistance frameworks do not effectively target or reach the extreme poor.

4. Grant and contracting mechanisms discourage adaptive programming.

5. Assistance frameworks hinder, rather than cultivate new forms of partnerships and market-based solutions that can reach the extreme poor at scale in fragile states.

[Read Mercy Corps’ recommendations in our memo “Managing Chaos”.]

When I got home from work the other night, the latest issues of Rolling Stone and The Economist magazines had arrived in my mailbox.   Of course I picked up Rolling Stone first to read about Melissa McCarthy, the break-out star of “Bridesmaids.” And that got me thinking.

In “Bridesmaids,” McCarthy was the antithesis of a hot Hollywood babe. But her character was immune to what others thought of her.  She was supremely confident, and in the end, she landed the guy she wanted.  So maybe I shouldn’t worry that the packaging of “management reform” isn’t seductive enough, at first glance, to make the QDDR II short list.  Maybe it has a shot after all.  All joking aside, I really hope it does because the future of the world’s poorest people depends on it.

Sampling IATI data round 1: Lessons learned

July 10th, 2014
Bookmark and Share

See below for a guest post from Ruth Salmon, Research Assistant at Publish What You Fund. Salmon is the lead on data collection for the 2014 Aid Transparency Index. This post originally appeared on Publish What You Fund’s blog on July 7, 2014.

***

Data collection continues for the 2014 Aid Transparency Index (ATI). And for the first time, we are sampling documents and data on the results, conditions and sub-national location published to the IATI Registry in XML format.

Sampling essentially means manually checking that the information provided is specific to that activity, has been appropriately tagged using IATI codes, and that it adheres to the ATI indicator guidelines.

The prospect of sampling documents and data for 16 ATI indicators, covering 25+ IATI standard elements, across 27 IATI publishers, was a daunting one. But we knew we had to plough our way through to ensure the information published about aid is relevant and of good enough quality to be used.

Yet this task turned out to be much more fun – and much less arduous – than we feared.

The good examples we found highlighted the potential of IATI data. There were plenty of things to get excited about. We’re happy to note in this first round of sampling that 82% of the 27 IATI publishers sampled passed the sampling checks for the indicators they publish.

The Good

As we clicked through the documents and IATI excerpts randomly selected for us by our sampling tool, there were many examples of good practice:

  • Subnational geo-coding: The European Commission’s Foreign Policy Instrument Services, the Netherlands, African Development Bank (AfDB)  and the World Bank are all publishing excellent geo-coded data, enabling us to see the coverage of project activities clearly within each country.drc-location
  • Results documents and data: Some results information clearly shows whether activities have achieved their intended outputs against stated goals. GAVI, Asian Development Bank (AsDB), Inter-American Development Bank (IADB), World Bank, and DFID have some great results documents, and Global Fund, Sweden and Canada  have some good examples of structured data.mcc-results
  • Appropriate language use: Several organisations (IADB, AfDB, UNDP) are publishing documents and IATI data fields using the language of the recipient country, making it more accessible for in-country users.

The Bad

Some less than helpful trends emerged too…

  • Over tagging: Some organisations are linking many documents to a single IATI category code, even when the code isn’t relevant to that document. This effectively renders the use of codes meaningless, making it very difficult to find the information you seek.
  • Inaccessible data: On a few occasions we were unable to access documents because of broken links. The use of scanned PDFs also makes the data difficult to parse and scrape. Document links led to generic web pages, with no clues on how to find documents for specific activities. Some samples were difficult to understand due to the acronyms and shorthand used to describe information.
  • Incoherence within data: Occasionally the codes used in the IATI data didn’t match the documents tagged. For example, country strategies tagged as organization strategies. There are several cases where the data specifies that no conditions are attached but conditions documents, with clear conditions outlined, are tagged!

The Ugly

Finally, a tiny percentage the IATI data we sampled was incomprehensible. Activities had no titles or descriptions, and were just lists of unlabelled transactions. This makes the information fairly useless to pretty much everyone. Queue many unrepeatable mutterings, long sighs and exasperated researchers.

The reality hit that it’s not possible to truly celebrate IATI publication until it is done well. Done badly, it serves very little use at all.

In summary…

Sampling has showed us the potential of using XML for publishing aid data for a wide variety of organisations. The elements of the IATI code had been used in many different ways, depending on what fits best with organisation’s particular structure and activities. When accurate and well-coded, it became clear that IATI data makes it easy to compare aid spending across time, space and many organisation types.

But it has also shown that unless the IATI standard is adhered to, the information published runs the risk of being difficult to understand, difficult to access, and difficult to reuse. Although it’s our impression that the majority of the errors we found were unintentional, it’s important that they are fixed to deliver on the full potential of IATI – of comprehensive, comparable aid data.

Now with data collection complete for 2014, we’re busy finalising and analysing the dataset so we can see this year’s results. Watch this space!

ForeignAssistance.gov Is Getting Bigger; Here’s How to Make It Better

June 25th, 2014
Bookmark and Share

See below for a guest post from Sarah Rose, Senior Policy Analyst at the Center for Global Development. The piece originally appeared on CGD’s blog on June 23rd.

***

We’re getting closer to knowing how the USG spends its foreign assistance dollars.  Recently, the State Department announced its first release of foreign assistance data on the ForeignAssistance.gov website (also known as “The Dashboard”).  This may not sound terribly glamorous, but it’s actually important news.  Since State’s spending makes up over a third of all US foreign assistance spending, the absence of its data has been a huge gap. With this recent State Department move, spending data for agencies responsible for 96 percent of US foreign assistance are now online. It’s great to see the Dashboard—now in its fourth year—slowly coming together. As it does, here are a few thoughts on why it’s still a good investment, the big challenges it faces, and how it can be improved.

Why We Should Cheer for the Dashboard

If well implemented, the Dashboard, an online resource of US foreign assistance spending (and potentially other) data, can:

  • Increase accountability and transparency: One of the Dashboard’s main goals is to enable easier access to information about US foreign assistance investments by US citizens, Congress, other US agencies, along with citizens and governments in recipient countries.
  • Ease agencies’ reporting burden (eventually): Behind the Dashboard lies a massive database that will eventually contain all of the underlying information necessary not just to populate the online interface but also to fulfill USG’s other regular reporting, like IATI, the Greenbook, and the OECD-DAC’s Creditor Reporting System.  Once the Dashboard/IATI process is automated within the agencies, complying with all this reporting should become much more streamlined and, importantly, more institutionalized.
  • Create incentives for improved data quality: Publishing data can change the dynamic around data quality.  The prospect of increased scrutiny can create an incentive for agencies to reinforce internal systems to produce cleaner, better organized data which can, in turn, bolster an agency’s own understanding of its internal operations.

Why It’s Taking So Long

The Dashboard was announced in 2010.  The effort is led by State’s F Bureau, which coordinates with the (over 20!) USG agencies that deliver some form of foreign assistance, and collects, codes, and publishes their data submissions. Some agencies, however, are far more capable of reporting to the Dashboard than others.  What’s so hard about data reporting, you may ask?  Quite a few things, it turns out, including:

  • Existing information systems’ incompatibility with Dashboard requirements.  Different agencies have different financial and project management information systems.  In fact, individual agencies often have multiple, separate systems.  Most of them long predate any notion of “open data” and are simply not designed to compile information in the way the Dashboard needs it.  Changing IT systems is a massive, costly undertaking.
  • Foreign assistance funds must be parsed out from a broader portfolio.  For agencies whose core mission isn’t foreign aid, internal systems weren’t set up to differentiate between foreign assistance and domestic spending. This makes it difficult to identify what’s right for the Dashboard and what’s not.  MCC has it easy in this respect (foreign aid only); the Department of Health and Human Services, for example, does not (mostly domestic).

At this point, the Dashboard team over at State is focused principally on providing data (i.e., getting more agencies on board) as well as pushing for improved data quality.  The team is pursuing a phased approach to populating the web portal, publishing agencies’ data as they have it ready.  It’s a courageous move for the USG to publicly release information knowing that it’s incomplete (and highly imperfect). Yet, they recognize that an incremental approach maintains pressure for continued implementation and fosters competition among agencies.  It may also help ease the culture shift towards transparency by gradually demonstrating that openness doesn’t have to be threatening.

Users Beware

This incremental approach also creates risks for users since:

  • A user can’t easily tell if data are complete—and often they’re not.  By illustration, this graphicshows agency-by-agency reporting to the Dashboard. You’ll see that not a single year contains information from all agencies (2006 to current), and that most agencies have reporting gaps.  It’s great that the Dashboard is frank about this, but the problem is that this is not clearly indicated where it needs to be.  For instance, if you wanted to find out about aid to Tanzania from 2008 to 2012, you would probably go directly to the Tanzania page and assume that what you pulled for “all agencies” means just that.  You’d be wrong. Only MCC and Treasury have 2008 data on the Dashboard, so “all agencies” means just those two for that year.  More broadly, it’s hard for a user to tell easily if data that don’t show up are absent because they don’t exist (e.g. DOD didn’t spend foreign assistance money in Country X in a given year) or because it’s missing (e.g. DOD did spend foreign assistance money in Country X that year but hasn’t reported it). The Dashboard does include caveats about data limitations but they’re unintuitively scattered in way too many locations that aren’t near where users are looking at data.  So they’re only helpful if a user thinks they should have a question about data quality or comprehensiveness and actively seeks this information.
  • Transaction-level data are incomplete (and sometimes unintelligible). Some important fields are missing from most agencies’ submissions.  For example, State is uniformly missing project title and description making it nearly impossible for a user to tell what he or she is looking at.  MCC has titles, but not descriptions.  USAID has descriptions for most of its transactions, but many of these merely replicate the title, are unintuitive to outsiders, refer to supporting documents that are unavailable, and/or cut off mid-description.  Start and end dates are also complicated.  For USDA they’re missing.  USAID provides only the year; MCC provides only the start date. State’s date reporting is spotty and contains apparently inconsistent information, like disbursements that happen before start dates.

Getting the data out there is important, and it’s the right thing to do.  But doing so while simultaneously improving coverage and quality gives me two related (though opposite) concerns.  I’m worried that:

1)      People Will Use the Data and draw incorrect conclusions due to missing or poor quality data; and/or

2)      People Won’t Use the Data because they are aware of its current limitations and will write off the Dashboard as an unreliable source, regardless of whether data coverage and quality improve later.  In a bit of a chicken and egg conundrum, lack of use could in turn slow Dashboard progress, since, to some extent, agencies need to know people will use the data before they invest scarce resources to provide it and improve its quality.

Ideas to Increase the Dashboard’s Potential

State’s Dashboard team and the 20+ agencies with foreign assistance spending are working hard to make the Dashboard a useful, relevant tool.  It’s a big undertaking.  Here are four things I hope they are considering:

1)      Help users better understand the data: The main risks to the Dashboard come from incomplete and thus unreliable data.  Breadth and reliability are key requirements for data to be truly useful. Therefore, the Dashboard should be abundantly clear when users are looking at complete versus partial information, or preliminary versus final data. Users should not have to dig through multiple, separate “additional information” pages to find this out.

2)      Improve transaction data:  Agencies should strive to fill the gaps in their transaction data (especially critical things like titles that facilitate rolling up transactions to the project level), as well as improve the comprehensibility of the information (for example, make descriptions descriptive).

3)      Don’t forget about usability: The current priority of the Dashboard is to publish as much data as possible in manipulable format and let users work with it as they wish.  However, a single user interface is never going to be able to meet the needs of all stakeholders, so the USG should reinforce its efforts to: (i) define who their priority audiences are; and (ii) understand how these different groups want to use the data and tailor the interface accordingly.  The Dashboard team is already taking steps in this direction with outreach to country missions and US-based stakeholders.

4)      Publish agency specific implementation schedules: The Dashboard website does explain where each agency is in the implementation process. But, it should also include agency-by-agency schedules for reporting compliance (and not just with Dashboard requirements, butwith IATI requirements, too).  This would not only provide an accountability structure that would help motivate continued momentum, it would also serve as an important signal of commitment.

Investing in Childhood: Building a Better Future for Mothers and Children in Afghanistan

June 19th, 2014
Bookmark and Share

See below for a guest post from Carolyn Miles, President & CEO of Save the Children and MFAN Co-Chair.

***

A few weeks ago, USAID Administrator Dr. Rajiv Shah wrote an op-ed in The Washington Post on how crucial it is to keep Afghanistan on the right track, so that the country can build upon the many gains and accomplishments already made. This will require continued investment, monitoring and accountability to ensure that short-term progress can evolve into long-term change for all Afghans, especially children and their families.

Save the Children has worked in Afghanistan since 1976. We have seen families struggle through decades of conflict and we know that an investment in children is the best way to build a better future for all Afghans. We have responded to droughts, floods and refugee crises in Afghanistan to help families when crisis strikes and we work every day to give children a healthy start, the opportunity to learn and protection from harm—things that every child deserves, no matter where they’re born.

In the past several years, it has become clear that our investment—and the investment of donor governments, global partners and Afghan leaders—is paying off in the lives of families. The increased availability of basic health services and training of community health workers who bring care to families in remote areas means better health outcomes, especially for vulnerable women and children.

In Save the Children’s annual State of the World’s Mothers report, for example, war-torn Afghanistan ranked as the toughest place on earth to be a mother in 2010 and 2011. This ranking is based on 5 key indicators: maternal mortality, child health, women’s educational opportunity, women’s economic status and women’s political representation—a lens through which to look at the experience of motherhood in countries around the world. Afghanistan’s place at the bottom of the list just a few short years ago speaks to the incredibly difficult circumstances in which Afghan mothers found themselves.

But in our latest report, released in May of this year, Afghanistan moved up an unprecedented 32 places in the ranking. This is thanks to the country’s investment in midwives, so mothers and newborns are safe at the dangerous time of birth; its dedication to providing lifesaving immunizations and other health interventions for newborns and young children; and its changing policies on education, so more girls can attend school.

Afghanistan has reduced maternal deaths by two thirds in 15 years, an almost unheard of accomplishment especially in a country experiencing ongoing conflict. In 2001, one of every four children born in Afghanistan died by the age of five; in 2014 that number is one in ten. These numbers are worth celebrating—it’s important to recognize the investment of so many partners who saw a reason to hope when the situation was bleak, and a reason to invest in children and mothers who deserved so much more.

Progress in education has been slower, especially for girls. Since 2002 the number of girls attending school increased by over 30%, although an estimated 1.5 million school-age girls are still not enrolled in classes.  Today, only 40% of Afghan girls attend elementary school and only one in 20 girls attend school beyond the sixth grade.  This is a huge area for improvement—and further investment can help Afghan girls access the best possible tool to build a better world for themselves and their families: education.

As Afghanistan and the world awaits the results of last weekend’s presidential run-off election, it’s more important than ever to look to the future.  Afghanistan is a potential success story of the power of foreign assistance—but it still has a long way to go. With smart aid, partner collaboration and remembering that an investment in a child is always a good investment, we can help Afghan children and families write the next chapter of the country’s remarkable progress.

More U.S. agencies publishing aid data to international standard

June 17th, 2014
Bookmark and Share

See below for a guest post from Publish What You Fund about the recent release of data by the State Department and the U.S. Department of Agriculture to the Foreign Assistance Dashboard.

***

In the last few weeks the Foreign Assistance Dashboard added two new U.S. agencies to its list of publishers.

The United States Department of Agriculture (USDA) published its aid information on May 28 and the State Department on June 1.

Publish What You Fund welcomes this progress, but initial analysis raises some very basic quality issues.

In summary, the Agriculture Department publication includes obligation and spending data from five of the nine USDA agencies implementing foreign assistance programs. The State Department’s data comes from the State’s centrally-managed financial system and includes both grants and contracts for domestic foreign assistance obligations and expenditures.

Both publications present significant data quality issues. For example, some activities lack a project name or project description. The start and end dates are equally problematic as in many cases they have the start and end date 01/01/0001 [State Department activities can be seen at the bottom of this file].

While Publish What You Fund encourages the publication of more foreign assistance from U.S. agencies, equally concerning is the quality of the data and what challenges it presents to the user of this information.

The Aid Transparency Index (ATI) data collection will close on June 30 and the data analysis will be conducted in July and August. Thorough analysis of this information, along with other U.S. agencies and programs’ data (including the Department of Defense, PEPFAR, MCC, Treasury and USAID) will be published in autumn.

Stay tuned for more developments as we near the launch of the ATI.

Dashboard-image1