Monique J. Beerli
Executive Director and Research Associate, Global Governance Centre
monique.beerli@graduateinstitue.ch
Twitter: @BeerliMonique
Synopsis: Is data the solution to achieving the 2030 Sustainable Development Agenda and fulfilling the 17 Global Goals? In this piece, Monique J. Beerli reflects on the practices and politics of measuring the Sustainable Development Goals, drawing insights from a high-level panel convened by the Global Governance Centre in November 2022.
Keywords: Sustainable Development Goals; 2030 Agenda; Data Practices; Global Governance by Indicators
Better data = better lives. This is now a formula known and pronounced by many, which undergirds the 2030 Agenda for Sustainable Development. For the United Nations Department of Social and Economic Affairs, collecting data “about the world and the people who live in it” is essential to assessing “what it takes to realize a better world for all.” More than just a technical matter though, concentrating data in the hands of the United Nations (UN) and wielding indicators as “technologies of global governance” raise a number of questions, some new and others old. If, as in the words of the Director of the UN Statistics Division Stefan Schweinfest, “what gets measured is what gets done,” what is to come of global problems that go unmeasured or are unmeasurable? With states now being assessed based on the performance of their statistical systems, as best illustrated by the World Bank’s Statistical Performance Indicators (SPI), to what extent is the push for internationally comparable data in the context of the 2030 Agenda generative of inter-state hierarchies? Moreover, is data for measuring the Sustainable Development Goals (SDGs) being collected, managed, used, and diffused in a fair, transparent, just, and respectful way so as to prevent the perpetuation of global inequalities and harm?
To shed some light on the multitude of issues that go hand in hand with the growing turn to data as a means of knowing and governing global problems, the Global Governance Centre convened a Measuring the SDGs: Data Practices, Challenges, and Futures roundtable in partnership with SDG Lab and Deloitte Switzerland. As a first exercise to consolidate a multistakeholder forum in International Geneva working in and around these critical themes, the roundtable brought together key representatives from international organizations, academia, civil society, and the private sector to reflect on SDG data practices and politics:
- Steve MacFeely, Director of Data and Analytics at the World Health Organization (WHO)
- Bojan Nastav, Acting Chief of Statistical Analysis at the United Nations Conference on Trade and Development (UNCTAD)
- Moira Faul, Executive Director and Senior Research Fellow at Network for International Policies and Cooperation in Education and Training (NORRAG)
- Kate Richards, Advocacy Manager of the Global Partnership for Sustainable Development Data
- Vincenzo Chiochia, Director and Head of AI Insights and Engagement at Deloitte Switzerland
As a follow-up to that event, this piece summarizes the main topics covered, beginning with a contextualization of UN’s turn to indicators.
Situating the UN’s Turn to Governance by Indicators
As has become common sense in the world of global governance and international affairs, in 2015, the United Nations General Assembly adopted the SDGs as a roadmap to achieving the 2030 Agenda. Presented as an international action plan “for people, planet, and prosperity” that equally promotes peace and partnerships, the 2030 Agenda introduced a total 17 Global Goals. Further sedimenting global goal-setting as a style of governing, the SDGs built upon and expanded the reach of the Millennium Development Goals (MDGs), both in terms of how development is defined and the scale of UN data demands. Whereas the MDGs focused on the eradication of extreme poverty, the SDGs cover an even wider terrain of global challenges and policy issues, namely inclusive social development, inclusive economic sustainability, as well as peace and security. Accordingly, the SDGs are far more ambitious and wide-ranging than the MDGs in terms of the goals and targets that they set.
Following the two-year process of intergovernmental negotiations and multistakeholder consultations that went into the making of the 17 Global Goals, the UN Statistical Commission took on the monumental task of translating political commitments into things that could be measured. With this mandate in hand, it formed the Inter-Agency and Expert Group on SDG Indicators (IAEG-SDGs) in 2015, which brings together representatives from member states as well as regional and international agencies. In 2017, the IAEG-SDGs introduced its Global Indicator Framework and, with that, the world witnessed the birth of 232 indicators, designed as a means to track and monitor SDG progress. Since, the United Nations’ thirst for data has intensified, turning data, and specifically quantitative data, into the apple of 2030 Agenda’s eye. Alongside a call to work with new types of data and as part of its internal strategy on new technologies, the UN is equally encouraging experimentation with machine learning and artificial intelligence techniques to accelerate the achievement of the SDGs. Currently, however, states are far from being able to satisfy the UN’s appetite for data.
Lacking Statistical Capacity and Data Incompleteness
In spite of the UN’s demands, most of the world’s countries lack the capacity to actually collect and communicate data on SDG indicators, thereby pointing to significant data gaps and weak statistical capacity. Ambitious in its goals, the SDG indicator framework represents an “unprecedented statistical challenge,” as underscored already back in 2016 by Mogens Lykketoft who served as the President of the seventieth session of the UN General Assembly. Far more complex and four and a half times more expansive than the MDGs, many of the 232 SDG indicators are entirely new, meaning that states do not have readily available data. According to some recent estimates on SDG reporting, no single country has data covering ninety percent of the SDG indicators, with most countries only having data on roughly fifty percent of the indicators. For the Global Partnership for Sustainable Development Data, approximately $650 million of additional funding per year would be needed to sufficiently boost SDG data capacities around the world. Properly measuring the SDGs is thus in and of itself a costly affair, as remarked by Moira Faul, Executive Director and Senior Research Fellow at NORRAG.
With many states around the world struggling to maintain their existing national statistical systems, the data demands of the SDG Global Indicator Framework only put greater strain on National Statistical Offices (NSOs). Whilst data may be the new gold, not everyone has the same capacity to extract and smelt that gold. For Steve MacFeely, the WHO’s Director of Data and Analytics, one of the biggest mistakes in transitioning from the MDGs to the SDGs was “not thinking enough about the impact on statistical systems and countries.” Keeping pace with the UN development agenda and having to amend statistical systems every fifteen years is simply a luxury that most NSOs cannot afford. With the expectation that a new development agenda will most certainly follow the SDGs, global policymakers should be mindful of the work required to generate SDG data and the frustration that would likely surface at the national level if states are presented with a new set of post-2030 indicators.
Whilst still endorsing policies geared towards developing the statistical capacity of states, the UN has begun to promote the potential value of unofficial alternative data sources, such as big data, geospatial data, and citizen-generated data, in tracking SDG progress. Increasingly private and commodified, alternative data sources now challenge the monopolistic power of states over data on their populations, thereby redrawing lines of power and data sovereignty. Relying more heavily on digital data as opposed to survey-generated data may amplify the data collection capacity of states, as remarked by Vincenzio Chiochia from Deloitte Switzerland. Talks of a data revolution, however, obfuscate the complicated realities and power dynamics behind the data.
Definitional Ambiguity
Alongside data incompleteness and weak statistical capacity, definitional ambiguity equally stands in the way of tracking and monitoring SDG progress. Indicative of such discrepancies, the IAEG-SDGs devised a three-tier classification system to distinguish between SDG indicators based on “their level of methodological development and the availability of data at the global level.” To qualify as Tier 1, indicators must be clearly conceptualized, have an internationally agreed upon standard for data collection, and already be measured by at least fifty percent of concerned countries. Tier 2 indicators have been defined and may potentially be but have not yet been measured based on a shared methodology. Tier 3 indicators, in contrast, remain conceptually ambiguous and lack an accepted data collection methodology.
In the case of Tier 3 indicators, United Nations bodies acting as custodian agencies play a vital role in facilitating conceptual and methodological consensus. Such was the case for Indicator 16.4.1 on illicit financial flows, as detailed by UNCTAD’s Acting Chief of Statistical Analysis Bojan Nastav. As part of Goal 16 on peace, justice, and strong institutions, member states agreed on making the reduction of illicit financial flows a target but failed to define it in clear terms. In their capacity as co-custodian agencies, UNCTAD together with the United Nations Office on Drugs and Crime (UNODC) convened a series of multistakeholder consultations with the International Monetary Fund (IMF), the Organization for Economic Cooperation and Development (OECD), UN regional commissions, and country experts to define illicit financial flows as well as how to measure them. Even in success stories, orchestrating definitional and methodological alignment at the international level is costly endeavor, which often excludes less powerful actors from data deliberations and produce policy blind spots.
Being Marginalized and Invisibilised by Data
Data can empower but can also exclude and harm. As Kate Richards from the Global Partnership for Sustainable Development Data asserted, populations are frequently solicited to give up data on themselves to powerful institutions, but don’t “have the right to determine how they are represented in data.” Nor do they have control over how and to what ends their data is utilized or how their data is then governed after being collected. Furthermore, datafication processes participate in further solidifying the status of certain social, economic, and political issues on the international agenda at the cost of invisibilizing and excluding others. Discussions on the missingness of SDG data therefore should not be limited to thinking in terms of missing data points. As emphasized by Moira Faul in the case of Goal 4 on inclusive and equitable quality education, SDG indicators allow “certain types, groups, and purposes of education data to be seen, which then allows the definition of a certain type of problem – and excludes others.” The widespread lack of disaggregated data, on education and beyond, is just one example of how vulnerable groups belonging to a specific age group, sex, socio-economic status, or other category are invisibilised. The overreliance on aggregated data then complicates the task of leaving no one behind as “vulnerable segments of the population remain hidden in the data.”
A reflection of the UN’s call for a data revolution, the 2030 Agenda for Sustainable Development has placed data at the forefront of managing global problems. According to the UN Statistical Division, in order to “fully implement and monitor progress on SDGs, decision makers need data and statistics.” At the heart of data-driven governance arrangements is the belief that data may solve collective inaction by subtly monitoring and putting pressure on states to ensure they fulfill their political commitments. While the promises of data are indeed enticing, technical initiatives designed with the intention of boosting the data capacities of states and international organizations, be it through surveys or big data, are inherently embedded in and reflective of power dynamics. Data is thus not only a force for good but also equally one that orders global society, excludes, and potentially harms, thereby necessitating deeper reflection on the politics of quantification and data-driven governance.