Blogpost written by Harvin Alexander Jordan Bello, DANE; Sabrina Juran, UNFPA; and Julia Manske, Data-Pop Alliance.
National Statistical Offices (NSOs) are undergoing a general paradigm shift, with digital technologies acting as main drivers of this development. They have lowered the costs of producing and publishing data, they have eased the distribution and visualization of data and hence democratized access to data. As a result, NSOs need to evolve on a continued basis from pure data producers to facilitators of comprehensible information that can be turned into knowledge.
In developing countries, many NSOs still struggle with elementary challenges; in Latin America for instance most notably with limited access to administrative records or the lack of local data. Nonetheless, the digital revolution offers great opportunities for them to leapfrog vital developments in the statistical process. In particular, the advent of big data could play a crucial role in this process. Big data promises to enable to get information about the most marginalized at higher timeliness and to this end promote evidence-based decision-making. However, the successful integration of big data in official statistical processes still lies far ahead of us and it comes with a set of ambiguities; such as methodological challenges, data protection concerns and technical settings.
To advance this process, it would be essential to develop strategies to modernize NSOs considering the need for redesigning statistical production systems as a starting point. These systems would integrate processes and sources with a production industrialization approach, which would enable the measurement of new phenomena that could be addressed from a multidimensional perspective. Furthermore, it would be key to strengthen the coordination of national statistical systems for the utilization of existing statistics.
Already today, many statistical offices are experimenting with big data. Many of them based in Latin America. They are setting up pilots, exploring the use of alternative data sources, such as Twitter and Facebook, georeferencing data entries or applying mobile technologies in data collection and analysis processes. For instance, at the office of DANE, Colombia satellite data has been used to complement the work of the survey staff of the national agricultural census.
NSOs will need to develop projects with exploratory dynamics to determine the extent and quality of new non-traditional sources of data for the production of statistics, taking into account compliance with internationally accepted quality standards and requirements. In addition, it is essential to identify the usefulness and mechanisms for the possible use of big data in topics where information gaps are evident, i.e. in the measurement of the Sustainable Development Goals. One way to do so could be by establishing „laboratories“ of innovation, ideally in collaboration with other partners.
To this end, openness towards new partnerships outside the community of official statistics will be key; with universities and academia to get access to talent and capacity and with other ministries, actors and organizations working with big data. In particular, public-private partnerships needs to be built. The private sector drives and stores large amounts of data in various spheres important to developmental and humanitarian work, knowing sometimes more about the fabric of societies than governments in the respective country. These partnerships will be even more important and more challenging in the developing context; where the supply of expertise is still low and the private sector is shaped by multinational players from the Western context. Hence, for NSOs in the Global South it will be decise to foster further cooperation between each other and install communities of practice.
In the long run, many would benefit from international agreements with private sector as well as from international guidelines and frameworks that will support these developments, for example when it comes to monitoring the SDGs.