This repository is designed to collect forecast data for the COVID-19 Forecast Hub run by the US CDC. The project collects forecast for the weekly new hospitalizations due to COVID-19. If you are interested in using these data for additional research or publications, please contact [email protected] for information regarding attribution of the source forecasts.
During the submission period, participating teams will be invited to submit national- and jurisdiction-specific (all 50 states, Washington DC, and Puerto Rico) probabilistic nowcasts and forecasts of the weekly number of confirmed COVID-19 hospital admissions during the preceding epidemiological week ("epiweek"), the current epiweek, and the following three epiweeks.
These weekly total COVID-19 admissions counts can be found in thetotalconfc19newadm column of the National Healthcare Safety Network (NHSN) [Hospital Respiratory Data] (HRD) dataset (https://www.cdc.gov/nhsn/psc/hospital-respiratory-reporting.html).
NHSN provides a preliminary release of each week's HRD data on Wednesdays here. Official weekly data is released on Fridays here. For more details on this dataset, its release schedule, and its schema, see the NHSN Hospital Respiratory Data page.
The Challenge Period is tentatively scheduled to begin with the epiweek of Sunday, November 17, 2024 with the first Forecast Due Date of November 20, 2024.
Participants will be asked to submit nowcasts and forecasts by 11PM USA Eastern Time each Wednesday (the "Forecast Due Date"). If it becomes necessary to change the Forecast Due Date or time deadline, CovidHub will notify participants at least one week in advance.
Weekly submissions (including file names) will be specified in terms of a "reference date": the Saturday following the Forecast Due Date. This is the last day of the USA/CDC epiweek (Sunday to Saturday) that contains the Forecast Due Date.
Participating teams will be able to submit national- and jurisdiction-specific (all 50 states, Washington DC, and Puerto Rico) predictions for two targets.
- Quantile predictions for epiweekly total laboratory-confirmed COVID-19 hospital admissions. This target is mandatory for any submitted location and forecast horizon.
- Individual forecast trajectories for epiweekly total laboratory-confirmed COVID-19 hospitalizations over time (i.e sampled trajectories). This target is optional for any submitted location.
Teams are encouraged but not required to submit forecasts for all weekly horizons or for all locations.
Teams can submit nowcasts or forecasts for these targets for the following temporal "horizons":
horizon = -1: the epiweek preceding the reference datehorizon = 0: the current epiweekhorizon = 1, 2, 3: each of the three upcoming epiweeks
As noted above, the source of this target data (epiweekly total admissions) will be NHSN; see the NHSN Hospital Respiratory Reporting page for data details. This dataset will also serve as the source of "truth data" for retrospective forecast evaluation. We also include a CSV file with latest values of incident covid-19 hospitalizations in the target-data directory.
We use epiweeks as defined by the US CDC, which run Sunday through Saturday. The target_end_date for a prediction is the Saturday that ends the epiweek of interest. That is:
target_end_date = reference_date + (horizon * 7)Standard software packages for R and Python can help you convert from dates to epiweeks and vice versa:
Detailed guidelines for formatting and submitting forecasts are available in the model-output directory README. Detailed guidelines for formatting and submitting model metadata can be found in the model-metadata directory README.
First-time pull requests (PRs) into the Hub repository must be reviewed and merged manually; subsequent ones can be merged automatically if they pass appropriate checks.
We suggest that teams submitting for the first time make a PR adding their model metadata file to the model-metadata directory by 4 PM USA Eastern Time on the Wednesday they plan to submit their first forecast. This will allow subsequent PRs that submit forecasts to be merged automatically, provided checks pass. We also request that teams sync their PR branch with the main branch using the Update branch button if their PR is behind the main branch, to ensure the automerge action runs smoothly.
We have made changes from previous versions of the COVID-19 Forecast Hub challenges to align COVID-19 forecasting challenges with influenza forecasting run via the Flusight Forecast Hub.
Both Hubs will require quantile-based forecasts of epiweekly incident hospital admissions reported into NHSN, with the same -1:3 week horizon span. Both will accept these forecasts via Github pull requests of files formatted according to the standard hubverse schema. The Hubs also plan to share a forecast deadline of 11pm USA/Eastern time on Wednesdays.
This repository follows the guidelines and standards outlined by the hubverse, which provides a set of data formats and open source tools for modeling hubs.