Business forecasting during the pandemic | SpringerLink

2.1

Data

In our main experiment, we take the perspective of an economist in the auto industry and simulate real-time forecasting for monthly total light vehicle sales (including autos and light trucks) in the United States. Data are available at monthly frequency from the Bureau of Economic Analysis starting in 1967 and accessed via the Haver Analytics database.

To test the generalizability of the results, we repeat the experiment for an outcome variable in another industry which may have experienced different supply and demand conditions during the pandemic: industrial production of information processing and related equipment (IP-IPRE). The category includes computers and peripheral equipment along with office, photocopy, communication and other related equipment. Data are available at monthly frequency from the Federal Reserve Board of Governors starting in 1967, accessed via Haver Analytics.

Summary statistics for both series are given in Table 1.

Table 1 Summary statistics for light vehicle sales (thousands) and IP: Information Processing and Related Equipment (index, 2017 = 100), Jan. 1967–Jan. 2022

Full size table

Figure 2 shows both series along with real GDP, with all three series rescaled so that 2018 equals 100. The figure shows that during the pandemic recession of 2020, both real GDP and IP-IPRE fell and then subsequently recovered, with a slightly more volatile recovery for IP-IPRE.

Fig. 2

Light Vehicle Sales, 1967–present and Industrial Production: Information Processing and Related Equipment, 1967–Present

Full size image

In contrast, the initial drop in light vehicle sales was much larger in percentage terms relative to the declines in GDP and IP-IPRE, consistent with consumers postponing big-ticket durable goods purchases during the recession. With the pandemic causing health concerns and shuttering many service businesses, along with pandemic fiscal support raising households’ disposable income, consumers shifted spending from services to goods beginning in summer 2020, and light vehicle sales experienced a sharper recovery compared to GDP. In the spring of 2021, semiconductor shortages and other supply chain bottlenecks weighed on automotive deliveries and production and sales fell again, with signs of a recovery starting in 2022. As a result, the auto sector’s recovery from the 2020 COVID recession has been markedly different from the recovery in broad GDP, which will have implications for the forecasting exercise described in the next section.

There are a couple of important caveats to our experiment when comparing against the real-world experience of a business forecaster. First, in real-time the forecaster is constrained by the economic data release schedule. For example, a business economist preparing a forecast at the end of March to support a sales planning meeting at the beginning of April will have readings for March’s consumer sentiment index, but consumer price index readings for March will not be available until the middle of April. In practice, many economists will use external forecasts or some statistical model to fill in this “jagged edge” of data. A second caveat is that the forecaster will also only have access to data as reported, while a number of series, such as nonfarm payrolls, can be revised significantly between the initial report and subsequent data releases.

In this article, we abstract from these data constraints, giving our business economist perfect foresight into the current month’s data (including future revisions), and thus an unfair knowledge advantage over a real-time forecaster. In the next section, we conduct a relative comparison of models that all benefit from this unfair advantage, rather than comparing the models against actual real-time forecasts published during the pandemic. Readers should note that the absolute forecasting performance of the models discussed below may be worse than depicted, due to these real-time data availability constraints.

2.2

Models

We consider a number of forecasting models commonly used in industry, described below. To conduct a real-time forecasting experiment, we use each model to generate out-of-sample forecasts for up to 12 months ahead, starting with sample data through December 2018 and making forecasts from January 2019 through December 2019. Moving forward in time, we then consider sample data through January 2019 and generate forecasts for February 2019 through January 2020, and so forth. At each step, we re-estimate parameters of the model and choose best-fit models according to relevant information criteria, so that a particular forecast model chosen in January 2019 may differ from the model chosen in December 2018. These forecasts are stored and compared versus actual observed outcomes to compare relative forecast accuracy.

2.2.1

ARIMA model (ARIMA)

The first kind of model we consider is a univariate autoregressive integrated moving average (ARIMA) model. For each sample step, we select an appropriate model order using the algorithm outlined in Hyndman and Khandakar (2008) and compute out-of-sample forecasts by iterating forward.

2.2.2

Vector autoregression models (VAR)

The second model we consider is a vector autoregression (VAR) which relates each variable in the system to lags of itself as well as lags of the other variables in the model, allowing it to capture feedback loops and interdependence between the endogenous variables. Following Stock and Watson (2002), along with the relevant outcome variable (annualized monthly growth of either light vehicle sales or IP-IPRE), we include monthly CPI inflation and the monthly change in the 90-day U.S. Treasury bill rate. The lag order of the VAR was chosen by selecting the order \(p \in \left[ {1,10} \right]\) which generated the lowest Bayesian information criterion (BIC). As in the case of the ARIMA model, for each sample step we compute out-of-sample forecasts by iterating the VAR forward.

2.2.3

VAR model with diffusion indexes (VAR-DI)

We also consider an alternative VAR model which incorporates information from a large set of macroeconomic indicators using the methods outlined in Stock and Watson (2002), rather than including a limited number of selected indicators. We use principal components analysis to extract the first two dynamic factors from a set of 181 monthly macroeconomic predictors available from January 1980 through January 2022. Stock and Watson (2002) interpret these factors as diffusion indexes measuring common movements in macroeconomic variables. Further details on the underlying predictor series are provided in Appendix 1. At each sample step using data through time t, we recompute factors using principal components analysis on the macroeconomic data. We then estimate a VAR model using these factors along with the relevant outcome variable: auto sales or IP-IPRE. We set the lag order and compute forecasts in the same way as the benchmark VAR.

2.2.4

h-step ahead autoregressive forecast (AR-Proj)

As an alternative to iteration-based forecasts, we could generate multistep forecasts directly by projecting future outcomes at time \(t + h\) onto data available at time \(t\). As a benchmark for this approach, we first consider a model that only uses lags of the variable to be forecasted to predict outcomes at time \(t + h\). The general forecasting equation is:

$$y_{t + h|t}^{h} = \alpha_{h} + \mathop \sum \limits_{j = 1}^{\rho } \gamma_{hj} y_{t – j + 1} + \varepsilon_{t + h}^{h}$$

(1)

For each sample step using data through time \(t\), we fit a forecasting model for horizons \(h \in \left[ {1,12} \right]\) with the lag order of the model \(\rho \in \left[ {0,6} \right]\) determined by BIC, where \(\rho = 0\) indicates that \(y_{t + h|t}^{h}\) is being projected onto a constant term only.

2.2.5

h-step ahead autoregressive forecast with diffusion indexes (AR-DI-Proj)

Another model specification we consider is an extension of the AR-Proj model that uses current and lagged values of \(y_{t}\), along with current and lagged values of the diffusion indexes, to predict \(y_{t + h}\). The forecasting equation is extended to reflect the diffusion indexes:

$$y_{t + h|t}^{h} = \alpha_{h} + \mathop \sum \limits_{j = 1}^{m} \beta_{hj}^{^{\prime}} F_{T – j + 1} + \mathop \sum \limits_{j = 1}^{\rho } \gamma_{hj} y_{t – j + 1} + \varepsilon_{t + h}^{h}$$

(2)

where \(F_{t}\) is the vector of factors whose estimation was described in the VAR-DI model section. At each step using data through time \(t\), we recompute factors using principal components analysis on the extended dataset of observables. We then fit a forecasting model for horizons \(h \in \left[ {1,12} \right]\) with the lag orders of the model \(\rho \in \left[ {0,6} \right]\) and \(m \in \left[ {1,3} \right]\) determined by BIC, where \(\rho = 0\) indicates \(y_{t + h|t}^{h}\) is being projected onto \(F_{t}\) and its lags (if selected) only.

2.3

Do high-frequency data improve forecasts?

As a final exercise, we conduct a simple test of whether incorporating high-frequency data would have improved forecast accuracy during the pandemic. To do this, we add an additional variable to the AR-Proj and AR-DI-Proj models that incorporates information from weekly high-frequency data. The regression equations take the form of:

$$y_{t + h|t}^{h} = \alpha_{h} + \mathop \sum \limits_{j = 1}^{m} \beta_{hj}^{^{\prime}} F_{T – j + 1} + \mathop \sum \limits_{j = 1}^{\rho } \gamma_{hj} y_{t – j + 1} + \delta W_{t} + \varepsilon_{t + h}^{h}$$

(3)

where \(W_{t}\) represents the high-frequency data variable, and \(\beta_{hj}^{{}} = 0\) for the version of the model that does not include the diffusion indexes.

The high-frequency variable we use is the Weekly Economic Index (WEI) published by the Federal Reserve Bank of New York (Lewis et al. 2020). The WEI represents the common component of ten different daily and weekly series, including same-store retail sales, unemployment insurance claims, a weekly staffing index, consumer surveys, steel production, electricity output, and other series. The series is scaled to four-quarter GDP growth units. Our regression uses the monthly average of weekly observations of the WEI, with data starting in January 2008, which shortens the available history of data relative to the other models under consideration. As a result, while incorporating high-frequency data might potentially help the forecaster identify upcoming turning points in the series, the shorter historical sample could also result in less precision when identifying the parameters of the statistical model and generating forecasts and prediction intervals.