Flexible Modeling of Epidemics with an Empirical Bayes Framework

• Accurate forecasts and advance notice of key events could help policy makers tailor countermeasures to each season’s epidemic behavior.Influenza epidemics occur annually and cause losses through lost productivity, sickness, and death.Source: §2 · p. 1
• The framework uses slightly modified past epidemics to generate possibilities for the current season without strong assumptions about disease propagation.The authors argue that models tailored to historical data may be too strict to reproduce current-season behavior.Source: §2 · p. 2
• The authors report forecasts for the 2013–2014 influenza season and retrospectively assess the framework’s accuracy.Source: §2 · p. 2

3 Introduction

• U.S. influenza’s annual burden is estimated at 611K lost undiscounted lifeyears, 3.1M hospitalized days, 31.4M outpatient visits, and $87.1B in economic burden.Forecasts can inform vaccine selection, vaccination campaigns and advisories, hospital staffing and beds, and planning by individuals and organizations.Source: §3 · p. 2
• The 2013–2014 CDC Predict the Influenza Season Challenge invited forecasts of epidemic progression and use of digital surveillance, and enabled assessment of prospective within-season forecasts.The competition also established a closer relationship between forecasters and policy makers.Source: §3 · p. 2
• Influenza curve models include compartmental approaches, agent-based models, and parametric statistical models such as generalized autoregressive moving-average models.Compartmental models estimate people in disease states; agent-based models simulate synthetic populations and interactions; parametric models range from linear autoregression to Box-Jenkins analysis and GLMs.Source: §3 · p. 2
• The framework modifies past seasons’ epidemic curves to allow changes in timing, pace, and intensity, while modeling observation noise and producing distributions rather than one-at-a-time point predictions.Unlike traditional nearest-neighbor modeling, it treats the entire season as a unit.Source: §3 · pp. 2–3

4.1.1 U.S. Outpatient Influenza-like Illness Surveillance Network (ILINet)

• ILINet measures ILI as the percentage of doctor visits related to the broader syndrome, since doctors do not generally diagnose influenza specifically and ILI is not generally notifiable.Source: §4.1.1 · p. 3
• More than 2,900 outpatient providers voluntarily report total and ILI-related visits, which CDC adjusts for participation changes and weights by state population to produce wILI.Source: §4.1.1 · p. 3
• CDC releases wILI weekly, with about a one-week reporting and processing delay, nationally and across ten HHS regions by age group; data may later be revised.Data are available from 1997–1998 onward; weeks 21–39 were missing in the first six seasons, while every week has been reported since 2003–2004.Source: §4.1.1 · p. 3

4.1.2 Google Flu Trends (GFT)

• GFT uses Google queries to estimate current-week ILINet activity in near real time, with final weekly estimates available soon after the week ends and estimates at national, regional, and smaller geographic levels.Source: §4.1.2 · p. 4
• GFT’s original 2008 algorithm was updated in 2009 and 2013 to regenerate queries and revise its method [21], despite criticism including performance versus simple alternatives.Source: §4.1.2 · p. 4
• At the competition’s outset, prior work considered GFT the most accurate existing digital surveillance system and helpful with ILINet; the study used it as a proxy when CDC data were unavailable or substantially revisable.Figure 1 illustrates the relationship between ILINet data, GFT data, and underlying phenomena.Source: §4.1.2 · p. 4

4.2 Empirical Bayes framework

• The framework smooths historical curves, transforms them into a current-season prior, estimates recent final wILI from non-final wILI and GFT, weights possible curves, and reports target forecasts.Source: §4.2 · p. 4
• Predictions were performed separately for the U.S. as a whole or individual HHS regions; data covered 15 historical seasonal influenza epidemics, but 2009–2010 was excluded for pandemic-related preseason behavior, and 2013–2014 data were partial.Source: §4.2 · p. 4

4.2.1 Data model

• Historical wILI trajectories are modeled as underlying ILI curves plus normally distributed noise, with quadratic trend filtering used to smooth each historical season.Source: §4.2.1 · p. 4

4.2.2 Prior

• The prior selects historical shapes and noise levels uniformly, while peak height and week use bounded uniform distributions and pacing stretches curves around their peaks.Source: §4.2.2 · p. 5

4.2.3 Sampling from the posterior

• Importance sampling weights transformed historical curves by their fit to observations from week 40 onward, then generates noisy future wILI trajectories; an alternative local transformation prior is under investigation.Source: §4.2.3 · p. 5

4.2.4 Forecasting targets

• For each weighted trajectory, the framework calculates onset, peak week, peak height, and duration, then represents their posterior distributions with histograms and reports posterior-mean point estimates.Source: §4.2.4 · p. 7

4.2.5 Incorporating non-final and digital data

• Forecasts used latest ILINet data alone or added GFT for weeks lacking ILINet data, optionally replacing recent ILINet values, with linear regression adjusting GFT estimates for non-logit-scale RMSE.Source: §4.2.5 · p. 7

5.1 Predictions for the 2013–2014 season

• Biweekly forecasts for the nation and 10 HHS regions reported posterior wILI curves and distributions for four targets, with posterior means as point predictions.The forecasts ran from December 5 (week 49) to March 27 (week 9), using revised data and no Google Flu Trends data in the displayed summary.Source: §5.1 · p. 8

5.1.1 Week 49 (December 5) forecast, using wILI data through week 47

• At week 49, forecasts concentrated onset predictions near the actual value, while wILI curves and the other three targets remained uncertain and closer to their priors.The onset point-prediction error was attributed in part to a sudden wILI jump at Thanksgiving, when fewer reporting-provider visits and holiday effects biased wILI upward.Source: §5.1.1 · p. 8

5.1.2 Week 1 (January 2) forecast, using wILI data through week 51

• By week 1, many posterior draws closely matched the observed wILI trajectory, duration predictions narrowed around the actual duration, and the confirmed onset became a point mass.The sudden peak was attributed to elevated ILI-related visits combined with fewer unrelated visits during winter holidays; posterior curves also included later peaks.Source: §5.1.2 · p. 8

5.1.3 Week 3 (January 16) forecast, using wILI data through week 1

• After the sudden peak, forecasts tended to latch onto transformed 2006–2007-like curves, degrading performance and continuing to predict a later peak until late-season forecasts matched the falling tail.Transformations expanded the set of reasonable curves given limited historical data, but did not eliminate latching.Source: §5.1.3 · p. 8

5.2 Point prediction trends

• Point-prediction errors included early peak underestimation attributed to unmodeled holiday effects and later peak-week, peak, and duration errors associated with latching onto past curves.The passage proposes additional prior curves, improved transformation selection, holiday adjustment, and more data as possible improvements, while favoring cross-validated historical error over tuning to one season.Source: §5.2 · p. 13

5.3 Estimated average error from cross-validation

• The empirical Bayes framework had competitive or lower mean absolute error than the best other baselines across many prediction weeks.Leave-one-out cross-validation estimated average error using final-revision wILI data; it did not account for approximating recent wILI from real-time tentative values.Source: §5.3 · p. 13
• The framework’s smooth distributions over curves and target values support point predictions for different losses, credible intervals, and probabilistic statements.Cross-validation compared the framework with baselines including historical means, conditioned trajectories, pointwise percentiles, k-nearest neighbors, and SIR-curve empirical Bayes.Source: §5.3 · pp. 13–14

6 Discussion

• Cross-validated errors indicate the forecasts compete with or improve on baseline predictors, and 2013–2014 influenza-season wILI forecasts looked reasonable to the human eye.The framework produces a distribution over future wILI curves and forecasting targets, not just point predictions.Source: §6 · p. 14
• The nonmechanistic framework generates epidemic curves from transformed historical data but provides no insight into the processes underlying an epidemic.It can be applied to other epidemics; the authors had used it to predict dengue incidence in 2014 World Cup host cities with little modification.Source: §6 · p. 14
• The framework could serve as a baseline for specialized predictors, and choices such as transformations could be selected automatically by minimizing historical cross-validated error.The paper expects applications to additional diseases to require little adjustment.Source: §6 · p. 14
• Potential improvements include combining non-final wILI with short-term regression, adding surveillance data, modeling holiday effects, and shrinking forecasts across nearby regions.The current framework uses ILINet and Google Flu Trends; the authors report historical spatial correlations among regional epidemics.Source: §6 · p. 14

National wILI Onset

• Figure 8 reports cross-validated mean absolute error estimates for national wILI onset point predictions, defined using the CDC’s 2% national threshold for the 2013–2014 season.Source: Figure 8 · p. 15

8.1 Full Forecast History, 2013–2014 Season

• Figures S1–S34 show the full 2013–2014 forecast history using the latest framework and final wILI values.For the Week 41 and Week 42 forecasts, observations from the forecast week and preceding week are not used.Source: §8.1 · p. 18, Figure S1 · p. 19, Figure S2 · p. 20
• The forecast displays include observed and future wILI, mean predictions, 95% pointwise bands, 10 posterior draws, and the CDC baseline.Source: §8.1 · p. 21
• Figures S28–S33 document national forecasts for Weeks 16–21, reporting point predictions, observed values, and absolute errors.“Pt Pred” is the posterior mean target value; “Obs” is the observed value, and “Abs Err” is the point prediction’s absolute error.Source: Figure S28 · p. 46, Figure S29 · p. 47, Figure S30 · p. 48
• Figure S34 continues the national forecast history with the Week 22 forecast and the same observation-exclusion rule.The forecast uses final wILI values and excludes observations for the forecast week and the preceding week.Source: Figure S34 · p. 52