Generally, big data increases an organization’s capacity to describe and visualise situations, diagnose and understand underlying problems, predict likely scenarios and even prescribe beneficial potential actions. In the data for development space, actors like UN Global Pulse and the Data Pop Alliance seek to promote a people-centred approach to big data, undertaking research and projects that use big data to solve persistent development problems, such as combatting malaria.

Although big data algorithms could generate new ways of looking at development and tackling problems, they are very much limited depending on which data is available. For example, algorithms that rely on mobile phone data will not be inclusive of people living in areas with low mobile phone penetration or poor network connectivity/availability as well as those less likely to own mobile phones (e.g. the poor, women, the elderly). Thus, taking a problem- and user-centric approach to big data for development – which is complemented by traditional data sources when possible – is essential to ensure that the right data from the right people is analysed to tackle the right problem.

To date, big data for development projects have mainly been pilots and proof of concepts showing the potential to use big data to tackle development problems. While these pilots have helped garner interest and hype around big data, so far there is no clear indication that they have led to data-based decision making. However, there is reason to remain hopeful as these pilots enable the development sector to start integrating long term big data applications into development programming.

The global open data movement began by advocating for openly available government data. For open data to make a real impact however, there are at least three key building blocks: the publication of open data by governments, the conversion of data to actionable information by intermediaries, and the use of this information by government officials, citizens, and others.

Open data became a primary advocacy target of the Open Government Partnership (OGP), whose OGP Open Data Working Group, composed of representatives from government and civil society, launched the International Open Data Charter in 2015, a manifesto and a commitment on how governments should publish government data. Other actors include the Open Data for Development Network, the Global Open Data for Agriculture and Nutrition, and loose alliances of researchers like the Open Data Research Network.

Case studies of open data initiatives in developing countries point to at least three emerging impacts: increased government transparency and accountability, innovation and economic development, and greater inclusion and citizen empowerment. These impacts, however, are dependent on a number of factors, the availability of open data being just one. It has become clear that open data impacts depend on partnerships and collaboration, the quality of public open data infrastructure, clear open data policies, and government responsiveness.

The idea behind citizen-generated data is for citizens to be more than passive consumers of information, but active data producers, users and intermediaries. While anecdotal, there is evidence of the value of citizen-generated data for improved delivery of development projects. Crowdmapping initiatives have contributed to a better understanding of the nature and scale of social issues such as corruption, harassment and voter turnout.

Key players in the field include Ushahidi whose open source software powers a great number of crowdmapping initiatives. Actors like the Humanitarian OpenStreetMap Team and Missing Maps that work across the globe to provide detailed and accurate maps for humanitarian response and economic development, have focused on maps as a tool in enabling citizens to provide data. In addition, several smaller initiatives have contributed to the advancement of the citizen-generated data field through practical experimentation with various forms of data collection techniques, tools and platforms.

Granular and real-time data provided by citizens have enabled governments to take more evidence-based decisions, for example about urban infrastructure. Moreover, research has documented the impact of citizen-generated data projects on government policy and practice in areas including measuring education outcomes and improving community water supply. However, citizen-generated data initiatives often face significant barriers to scaling, including requiring a robust data collection infrastructure and ensuring that data production is sustainable and citizens are committed to contributing data.

Digital real-time data requires employing digital technology to enable and accelerate collection, sharing, management, analysis, and reporting of data to enable timely decision-making. The expected impacts of using real-time data in development programs are related to increasing their capacity to respond to changes in their operating contexts and to learn from constant evaluations of the effectiveness of their actions. Real-time data has the potential to give development actors the means to uncover anomalies, respond to issues as they arise, improve internal coordination, optimize resource allocation, react to citizen feedback and anticipate trends and future events.

Not many development organisations have initiatives focusing on real-time data. However, there are a number of development projects that use digital technologies and rely on real-time data in one way or another, even if they do not consider themselves to be real-time data projects. For example, UNICEF has supported the development and expansion of several open source real-time data systems, including RapidFTR, RapidSMS and RapidPro. One of UNICEF’s flagship initiatives is U-Report, a mobile social messaging tool used in 35 countries, which allows communities to share what they care about and their sentiment towards development issues and programmes through polls. GIZ, on the other hand has supported the development of the open source platform SORMAS, a disease outbreak early warning and management system that was used during the West African Ebola outbreak in 2014/15.

Nevertheless, there is still little evidence that real-time data leads to adaptive decision making, as there are many barriers to ensuring data is used by decision makers. Some of the hindrances include competing data sources, distrust in data quality, lack of awareness of the data, data not adequately turned into information, lack of visualisation, and information that is not tailored to the end user’s needs.