Skip to main content
U.S. flag

An official website of the United States government

Evaluation of the US COVID-19 Scenario Modeling Hub for informing pandemic response under uncertainty

November 20, 2023

Our ability to forecast epidemics far into the future is constrained by the many complexities of disease systems. Realistic longer-term projections may, however, be possible under well-defined scenarios that specify the future state of critical epidemic drivers. Since December 2020, the U.S. COVID-19 Scenario Modeling Hub (SMH) has convened multiple modeling teams to make months ahead projections of SARS-CoV-2 burden, totaling nearly 1.8 million national and state-level projections. Here, we find SMH performance varied widely as a function of both scenario validity and model calibration. We show scenarios remained close to reality for 22 weeks on average before the arrival of unanticipated SARS-CoV-2 variants invalidated key assumptions. An ensemble of participating models that preserved variation between models (using the linear opinion pool method) was consistently more reliable than any single model in periods of valid scenario assumptions, while projection interval coverage was near target levels. SMH projections were used to guide pandemic response, illustrating the value of collaborative hubs for longer-term scenario projections.

Publication Year 2023
Title Evaluation of the US COVID-19 Scenario Modeling Hub for informing pandemic response under uncertainty
DOI 10.1038/s41467-023-42680-x
Authors Emily Howerton, Lucie Contamin, Luke C Mullany, Michelle Qin, Nicholas G. Reich, Samantha Bents, Rebecca K. Borchering, Sung-mok Jung, Sara L Loo, Claire P. Smith, John Levander, Jessica Kerr, J. Espino, Willem G. van Panhuis, Harry Hochheiser, Marta Galanti, Teresa K Yamana, Sen Pei, Jeffrey L. Shaman, Kaitlin Rainwater-Lovett, Matt Kinsey, Kate Tallaksen, Shelby Wilson, Lauren Shin, Joseph C Lemaitre, Joshua Kaminsky, Juan Dent Hulse, Elizabeth C. Lee, Clif McKee, Alison Hill, Dean Karlen, Matteo Chinazzi, Jessica T Davis, Kunpeng Mu, Xinyue Xiong, Ana Pastore Piontti, Alessandro Vespignani, Erik T Rosenstrom, Julie S Ivy, Maria E Mayorga, Julie L Swann, Guido España, Sean Cavany, Sean Moore, Alex Perkins, Thomas J. Hladish, Alexander N. Pillai, Kok Ben Toh, Ira Longini Jr., Shi Chen, Rajib Paul, Daniel Janies, Jean-Claude Thill, Anass Bouchnita, Kaiming Bi, Michael Lachmann, Spencer Fox, Lauren Ancel Meyers, Ajitesh Srivastava, Przemyslaw Porebski, Srinivasan Venkatramanan, Aniruddha Adiga, Bryan Lewis, Brian Klahn, Joseph Outten, Benjamin Hurt, Jiangzhuo Chen, Henning Mortveit, Amanda Wilson, Madhav Marathe, Stefan Hoops, Parantapa Bhattacharya, Dustin Machi, Betsy L Gunnels, Jessica M Healy, Rachel B. Slayton, Michael A Johansson, Matthew Biggerstaff, Shaun Truelove, Michael C. Runge, Katriona Shea, Cécile Viboud, Justin Lessler
Publication Type Article
Publication Subtype Journal Article
Series Title Epidemics
Index ID 70250205
Record Source USGS Publications Warehouse
USGS Organization Eastern Ecological Science Center