Are we prepared for the next pandemic? New tools for measurement and assessment
Published in Public Health
Since the beginning of the pandemic, a number of analyses have suggested that standard preparedness measures did not predict COVID-19 mortality or other outcomes. These studies investigated the WHO’s States Party Self-Assessment Annual Reporting (SPAR) tool and the Joint External Evaluation (JEE) tool and the Global Health Security Index (GHSI). Such analyses led the Independent Panel for Pandemic Preparedness and Response to say that “the failure of these metrics to be predictive demonstrates the need for a fundamental reassessment which better aligns preparedness measurement with operational capacities in real-world stress situations, including the points at which coordination structures and decision-making may fail.”
Having studied tools for measuring and assessing public health emergency preparedness, we were concerned that the first part of the Independent Panel’s conclusion – about the failure to be predictive – was misleading. The analyses are essentially asking the wrong question; evaluating metrics involves more than asking whether they predict outcomes.
Nevertheless, we agree about the need for better measurement tools that capture operational capabilities, including coordination and decision-making. Understanding the distinction requires taking a measurement science perspective. This well-established paradigm begins with the fundamental question of “why measure,” with accountability, quality improvement, and resource allocation and mobilization as the most common uses. Answers to this question, in turn, help frame approaches to the remaining questions: “what to measure,” “how to measure” (i.e., collect data to populate measures), and assessing “how well measures work” (i.e., their validity, reliability, feasibility, and utility).
Our recent paper in Globalization and Health, identifies two reasons why standard measures failed to predict COVID-19 outcomes. First, there are statistical problems: the quality and comparability of the data across countries, the focus on simplistic outcome measures, and the inadequacy of simple cross-sectional study designs call into question the validity of these analyses.
What to measure: But there also is a serious conceptual issue: conflating resilience and preparedness. Various communities of practice use the terms in different ways, but we think of resilience as the ability of individuals, communities, and systems to adapt and recover from adversity, including public health emergencies. From this perspective, current preparedness measures fail to predict pandemic outcomes because they do not adequately capture variations in the presence of effective political leadership needed to activate and implement existing systems, instill confidence in the government’s response or differences in interpersonal and institutional trust needed to mount fast and adaptable responses.
Preparedness metrics, on the other hand, are not intended primarily to predict outcomes. Rather, their purpose is to hold countries accountable for their obligations under the International Health Regulations and to identify gaps in preparedness systems so they can be addressed and improved. Thus, the validity of the SPAR and JEE tools, and the GHSI depends on how well these metrics identify gaps in preparedness capacities and capabilities that are necessary, but not sufficient, to guarantee good outcomes. Although governance and trust are critical, the international community can only hold countries accountable for their treaty obligations.
While our paper was in review, Jorge Ledesma and colleagues published an analysis that addressed both of our concerns. First, related to our technical concerns, they used age-adjusted excess mortality estimates with data over two years (2020 and 2021) rather than a limited window as the outcome measure. Second, with this statistical approach they found quite different result than the other analyses: higher GHSI scores were associated with lower COVID-19 deaths, as one would expect from a good measure of preparedness.
Moreover, Ledesma and colleagues further analyzed the results in a multivariate regression exploring the impact of the six components of the GHSI. Controlling for the other components, they found that the “risk environment score” had a stronger relationship to COVID-19 mortality than any of the others. This score includes government effectiveness, public confidence in governance, trust in medical and health advice, and related factors. The other five components are more traditional preprepared measures; the risk environment score at least partially includes governance and social trust, which are necessary for a successful response.
However, although our paper explains why preparedness measures didn’t predict COVID-19 outcomes, we agree with the Independent Panel’s call for better measurement tools better aligned with operational capacities in real-world stress situations. This is where measurement science’s consideration of “what to measure” and “how to measure” comes in.
What to measure: ensuring accountability under the IHR requires going beyond summary JEE or GHSI scores to identify specific gaps in operational capabilities. One lesson from the pandemic, for instance, was that the standard measures did not capture all of the relevant constructs. The SPAR tool and the GHSI have both been updated since to reflect lessons learned in the pandemic about, for instance, the need for enhanced surveillance. Similarly, the European Union’s template for triennial reporting on preparedness in the Member States now reflects the need to scale-up testing and laboratory efforts, not just the availability of surveillance systems.
How to measure: we can’t improve systems without knowing how they operate and why they might fail. This requires qualitative assessments as well as quantitative indicators, and fortunately, several tools have been developed in recent years to address this need. Rigorous methods for After- and Intra-Action Reviews have been developed and were effectively used to examine how systems performed under stress during the pandemic. Simulation exercises and stress tests have also been developed as tools to assess preparedness capabilities.
Along these lines, Prevent Epidemics has developed the 7-1-7 process as an organizing principle, target, and accountability metric. The process focuses on three specific time milestones: 7 days to detect a suspected outbreak, 1 day to inform national public health authorities to initiate an investigation, and 7 days to initiate an effective response. The idea is that continuously evaluating and improving timeliness can identify performance bottlenecks and help to improve detection speed and response quality. WHO is currently working to incorporate this approach, which they label Early Action Review, with a OneHealth focus.
No single score – or indeed any particular tool – can capture every important dimension or perfectly predict outcomes. But taken together, new tools for measuring and assessing public health emergency preparedness offer opportunities to ensure countries’ accountability to one another and to improve the global, national, and local preparedness systems that are essential for improving global health security.
Follow the Topic
-
Globalization and Health
This is a pioneering transdisciplinary journal that situates public health and well-being within the dynamic forces of global development.
Related Collections
With Collections, you can get published faster and increase your visibility.
From Aid Dependency to Health Sovereignty? The Political Economy of Global Health Financing in the Post-COVID Era
The COVID-19 pandemic exposed deep inequalities in global health governance, but it also intensified a longer-running political economy problem: how health systems, preparedness capacities, and cross-border health cooperation are financed, by whom, and on whose terms. In the post-COVID period, ambitions for pandemic preparedness, universal health coverage, regional resilience, and health sovereignty are unfolding amid debt distress, fiscal consolidation, donor retrenchment, aid fragmentation, and renewed debate over the legitimacy and effectiveness of established global health partnership models.
For the purposes of this Collection, health sovereignty refers to the capacity of countries and regions to exercise meaningful authority over health priorities and systems, including by setting priorities in ways accountable to their populations, aligning external support with those priorities, and addressing power asymmetries in global health financing and governance. It does not imply self-sufficiency or disengagement from international cooperation.
This Collection contributes to a wider Super Collection across Globalization and Health, Infectious Diseases of Poverty, and One Health Outlook. The collaboration brings together complementary perspectives on the changing global health financing landscape: political economy and governance, infectious disease control in low-resource settings, and the financing of integrated human, animal, and environmental health systems. Together, the three Collections offer a cross-disciplinary account of how geopolitical and economic change is shaping preparedness, equity, resilience, programme sustainability, and the future of global health governance.
This specific Collection invites contributions that examine the changing political economy of global health financing after COVID-19. Rather than treating global health financing as merely a technical question of resource mobilisation, the Collection adopts a political determinants of health perspective, asking how financing arrangements allocate authority, constrain or expand policy autonomy, reshape institutional power, affect health system resilience, and reproduce or reduce inequalities between countries and regions. It is particularly interested in whether contemporary shifts reduce aid dependency and strengthen health sovereignty through national, regional, and more publicly accountable financing arrangements, or instead reproduce older dependencies in new forms.
Indicative topics
- The Collection welcomes theoretically informed and empirically grounded contributions on topics including, but not limited to:
- The politics of mobilising and allocating resources for health in contexts of debt vulnerability, constraints on fiscal space, austerity, and external policy influence, including the implications for health-system financing, preparedness, and infectious disease programmes;
- The role of the IMF, World Bank, regional development banks, pandemic funds, global health partnerships, and other multilateral institutions in shaping post-COVID health financing; how these organisations are adapting or reforming in response to aid retrenchment and demands for greater health sovereignty; and how their financing arrangements influence national or regional priority-setting;
- Processes of regionalisation in global health financing, including the development of regional and interregional cooperation and financing mechanisms across different regions and regional institutions (e.g., PAHO, ASEAN, AU, EU); their implications for health sovereignty and global health governance; and the ways in which regions cooperate or compete, including through interregional partnerships (e.g., EU–AU cooperation);
- Commercial, private, and philanthropic financing in global health, including blended finance, development finance, bond markets, and private creditor power; the growing role of philanthropic foundations in filling funding gaps and setting priorities; and the governance implications of conditional grants, strategic investments, royalty arrangements, and preferential rights over resulting intellectual property;
- Vaccine manufacturing, supply chains, technology access, and procurement, provided that financing, dependency, institutional power, or regional autonomy is a central analytical concern.
Approach and submission requirements
The Collection is especially interested in submissions that bridge global health scholarship and international political economy. Original research articles may draw on quantitative, qualitative, comparative, historical, conceptual, or policy-oriented approaches. Submissions should treat financing, resource flows, fiscal authority, institutional power, dependency, or sovereignty as a central analytical concern rather than as background context. They should examine transnational dynamics or, where centred on a national or regional case, demonstrate its wider transnational implications.
Topics outside the scope
To keep the Collection focused, submissions will fall outside the scope if they primarily:
- present clinical, epidemiological, health-services, or technical or operational research—including on vaccine supply chains, pharmaceutical intellectual property, biomedical innovation, procurement, or disease-specific programme delivery—in which financing and transnational political economy are peripheral;
- present technical health financing analysis, such as costing, actuarial modelling, economic evaluation, provider-payment design, insurance reform, or budgeting, without a substantive political economy argument and transnational dimension;
- examine aid, debt, fiscal policy, development finance, or international institutions only at a general level, without sustained analysis of their implications for health financing, health systems, or preparedness;
- address health security, pandemic preparedness, health diplomacy, geopolitics, regional cooperation, or global health governance without substantive analysis of how financing arrangements or resource flows shape the issue;
- focus on a country’s domestic politics or policies without analysing their connections to transnational actors, policies, processes, or initiatives;
- primarily describe, evaluate, or promote a particular global health initiative, funding mechanism, organisational model, institutional strategy, or aid programme without using the case to advance a broader analytical argument about financing, power, dependency, sovereignty, or institutional change;
- focus on equity, decolonisation, or the organisation of North-South academic and research partnerships without treating financial relationships or resource flows as a central part of the analysis; or
- offer a mainly descriptive mapping of funding flows or post-COVID global health governance without a clear analytical argument about how financing arrangements shape power, autonomy, accountability, equity, health system resilience, or preparedness.
Article types
The open call will accept original research articles only. Submissions led by scholars based in the Global South, as well as equitable collaborations involving Global South institutions, are particularly encouraged. Commentaries may be commissioned separately and will not be accepted through the open call.
All submissions in this collection undergo the journal’s standard peer review process. Similarly, all manuscripts authored by a Guest Editor(s) will be handled by the Editors-in-Chief. As an open access publication, this journal levies an article processing fee (details here). We recognize that many key stakeholders may not have access to such resources and are committed to supporting participation in this issue wherever resources are a barrier. For more information about what support may be available, please visit OA funding and support, or email OAfundingpolicy@springernature.com or the Editors-in-Chief.
Publishing Model: Open Access
Deadline: Jun 04, 2027
