1 Introduction

The need for a comprehensive set of macroeconomic statistics has become increasingly important nowadays, a period strongly characterized by acute developments in modern societies. This demand is particularly directed towards to national statistical institutes (NSIs) with attention to expanding the publication of official short term economic indicators, rapid estimates of quarterly accounts, as well as monthly estimates of GDP (Pfeffermann 2015; Jarmin 2019; Frale et al. 2010, 2011; Grassi et al. 2015; Proietti et al. 2016). In this respect, Eurostat has made several efforts in Europe to develop specific handbooks and guidelines on methods for both rapid estimates and quarterly national accounts (Eurostat 2017b, 2013).

For instance, in Italy, the official turnover index in services is still available only at a quarterly frequency and with a delay of around 55 days from the end of the quarter. This presents a big challenge for quarterly accounts, particularly for the exercise carried out at \(t+30\) days after the end of each quarter, when the preliminary GDP estimate is released. In practice, the last observation of the reference indicator for market-service value added estimation remains in large part uncovered and the usual solution is to proceed by means of ARIMA forecasting. The risk is, of course, a large revision of the GDP estimate on the occasion of its second release.

Another point is that we are in the age of big data. We cannot hide the explosion of alternative indicators which are available also at high frequency, either free or at relatively low cost. NSIs have started to use these data for official statistics, such as scanned price data in supermarkets as an indicator of inflation in several countries, VAT returns as an indicator of GDP components, and other relevant applications (Feenstra and Shapiro 2007; Labonne and Weale 2020). In our view, soft data coming from business surveys officially conducted by NSIs can also be useful. These data are built as proxies of short-term indicators, are available monthly and rapidly, and their use is well established for the production of coincident or leading business cycle indicators (Eurostat 2017a).

Given the large number of alternative indicators, one of the problems that arises is selection. One option is to limit the choice to qualitative considerations based on economic sense, for example, the use of the climate index in transport from business surveys for disaggregation/prediction of turnover in the same sector (Glotain and La Tente 2016).

Another option is to make a selection based on standard correlation indexes between the target and the set of alternative indicators or other classical approaches suggested in Ng (2013).

Yet another option is to extract a few factors from the alternative indicators by means of statistical methods such as principal component analysis. All these procedures require a second step in which the selected indicators are included in a model to estimate and forecast the target variable (see, among others, Boivin and Ng (2005), Bańbura et al. (2010), Cubadda and Guardabascio (2012) and the literature cited therein).

A shrinkage method such as Least Absolute Shrinkage and Selection Operator (LASSO) can also be used. It allows one to simultaneously select important indicators and estimate their impact while being guided by the prediction of the target variable (Tibshirani 1996; Belloni et al. 2011; Lim and Hastie 2015). Note that to consistently estimate the coefficients we need to take care of the mixed frequency structure of the available data and try to disaggregate the target series in order to use information coming from high frequency indicators. In this respect, Proietti et al. (2021a) and Proietti et al. (2021b) provide a very elegant solution to the ‘big-data’ challenge for monthly estimations of, respectively, the euro area and the Italian GDP. Here, a two-step approach is applied: first, a bivariate dynamic factor model is fitted to the quarterly GDP and one monthly indicator at a time; then, the estimates are combined using weights, which reflect the ability of each indicator to nowcast quarterly GDP. The weights are obtained as a function of the regularized estimator of the high-dimensional covariance matrix of the nowcasting errors.

The aim of this paper is to show the merits of forecasting and distributing at a monthly frequency a quarterly time series, using a selection of monthly related indicators extracted from a large set of potential predictors of the quarterly aggregate. A mixed frequency approach based on sparse temporal disaggregation by Mosley et al. (2022) is applied, which extends the classical regression-based Chow and Lin temporal disaggregation family (Chow and Lin 1971) by including a LASSO regularization framework that allows a high-dimensional set of regressors to be included in the model. Through simulation studies, Mosley et al. (2022) demonstrate that this provides a great improvement in disaggregation performance and parameter estimation over Chow and Lin in both moderate- and high-dimensional cases.

The method of Mosley et al. (2022) is very suitable for the task at hand, since it provides a compact one-step framework to find higher-frequency estimates of the target variable, select a few of the most important indicators from a large set, and obtain out-of-sample forecasts. As opposed to the two-step methods described above, the dimensionality reduction is not performed before the forecasting, but is guided by the forecasting, and thus potentially important information is not discarded. Using the method of sparse temporal disaggregation, our results prove both an improvement in predictive accuracy over other methods and an interpretability into significant indicators when forecasting a date of interest.

Its simplicity is also one of the main merits of the approach of Mosley et al. (2022), a feature particularly relevant for the massive production of disaggregated data in the context of official statistics, where both seasonally adjusted and unadjusted data are always required (see, among others, Eurostat (2014) for more details on national accounts and Eurostat (2021) for short-term statistics). Of course, the literature sees the practice of deriving seasonal disaggregated series from unadjusted indicators as a statistical artefact, since the low-frequency aggregate does not contain information over the higher-order harmonics of the seasonal pattern which, therefore, are simply ‘borrowed’ from those of the indicator series. In the Chow and Lin type regressions, this simplification implies that the seasonal component of disaggregated data over the frequencies which are unknown for the dependent variable are proportional to those of the indicator. Details are provided in Proietti and Moauro (2008). Given that this contrast appears irreconcilable, we carry out the exercise on both seasonally adjusted and unadjusted data, highlighting the merits and limitations of the two approaches step by step.

The proposed method is applied to a real-time study of the turnover in Italian transport, which is particularly challenging since this was greatly affected in 2020 by the dramatic movements due to the COVID-19 pandemic and the resurgence of inflation at the end of 2021. Potential monthly indicators are taken from a total of 143 time series: 56 series of business surveys in transport about both the climate and frequency of responses, 18 series from Assaeroporti about both passenger and cargo flights split by national and international routes, and 69 series of monthly turnover in industry split by both sector of economic activity and reference market. The sample spans from January 2010 to December 2021 for both seasonally adjusted and unadjusted data. Several aspects of the estimation are considered: the stability of the selected indicators over the quarters 2017–2021; the reliability of the estimates in terms of their monthly pattern; their forecasting performance.

The case study here presented is of great importance, since it reflects situations of lack of data (either data published with a delay, or at low frequency of observation, etc) typical of many countries in the economic activities of the service sector. This is a sector characterized by broad heterogeneity from one branch to another and, for this reason, requiring specific surveys for the collection of the data. Moreover, the exercise is easily replicable since implemented under R (code available upon request) and the data are all in the public domain.

The remainder of this paper is organized as follows: The next section explores the key reasons for studying the transport sector. Section 3 presents the full dataset. Section 4 illustrates the model. Section 5 describes the empirical application and its main results. Section 6 is devoted to discussion. Finally, Sect. 7 draws some conclusions.

2 Evolution of the transport sector across time

The significance of services in the Italian economy, as well as in other countries, is evaluated by measuring the value added of this sector as a percentage of GDP. From 1990 to 2022, Italy’s average value for this indicator stood at 64.18%, with a minimum of 60.11% in 1990 and a maximum of 67.09% in 2014. In 2022, the latest value is 64.79%, while the global average for the same year, based on data from 188 countries, is 53.18% according to The World Bank (2022). These data provide us with valuable information on the role of services in the Italian economy and its comparison with other countries. Indeed, the consistently high average of this indicator for Italy shows the significant contribution of services to Italy’s overall economic output. The fluctuation between the minimum and maximum values over the years reflects the dynamic nature of the sector and its ability to respond to economic conditions. Furthermore, the global average of 53.18% for this indicator in 2022 highlights the important role of services in driving economic growth and development worldwide and the importance of the turnover index in services as a crucial indicator of the business cycle, showing the evolution of turnover in the service industries. However, there is a concern about getting timely estimates, which can also result in revisions from the GDP estimate. This poses a risk to the accuracy and reliability of the data because the process of developing official indicators by NSIs is demanding and often does not cover all the phenomena under study.

The turnover index in services, classified according to the NACE Rev. 2 system, covers a wide range of activities such as wholesale (G) transportation and storage (H), accommodation and food services (I), information and communication services (J), real estate services (L), professional, scientific and technical activities (M), and administrative and support services (N). Among them, in recent years, transportation and storage have gained significant attention from transportation planners, economic development specialists, and private industry.

The globalization of the economy, the consumer-oriented market (Strasser et al. 1998), Internet-based information systems, the reduction of trade barriers (McCann 2008), tariffs, and transportation costs (Rodrigue and Notteboom 2009), along with the European traffic policy, have led to a significant increase in global trade. This has created intense international competition, causing firms to focus on meeting consumer demand, delivering goods quickly, reducing costs, and improving quality (Bonacich and Wilson 2008). Companies are using techniques of international logistics to gain a competitive edge in managing supply chains (Long 2003). The introduction of the global value chain has fragmented the production processes and spread tasks across borders (Gereffi et al. 2005). Communication, information flows, and logistics are crucial in governing the global network of transnational companies. Logistics plays a vital role in reducing delivery times, increasing reliability and flexibility, and improving the response to the customers (Barlow 2004), which benefits all parties in the supply chain. As a result, companies are evaluating the availability, quality and cost of transportation services (Bonacich and Wilson 2008) and are focusing on the geographical distribution of freight (Hesse and Rodrigue 2004). The establishment of major logistics parks along intermodal hubs is seen as a vital component of the industrial economy to ensure thriving freight activity and a reliable source of income.

Several European countries have implemented policies to promote competition in the transportation sector and address changes in logistics. These policies focus on the development of infrastructure, regulatory measures, trade policies, subsidies and incentives, taxation, and funding. The development of infrastructure aims to reduce transportation costs and improve efficiency (Beyzatlar et al. 2014; Berechman et al. 2006; Farhadi 2015; Saboori et al. 2014). Regulatory policies impose entry barriers and price restrictions, reducing overall economic growth (Alesina et al. 2005; Barone and Cingano 2011). Reforms in previously protected industries lead to increased investment. Trade policies and agreements affect the volume of goods transported (Ietto-Gillies 2015). Incentives and taxation policies promote specific modes of transportation, influencing consumer behaviour (Glaeser and Kohlhase 2004). Government policies have a significant effect on the transportation industry, influencing demand trends, operating costs, and overall industry dynamics. By implementing well-designed and strategic policies that promote efficient and sustainable transportation systems, governments can contribute to long-term economic progress (Haque et al. 2013). However, policies that are poorly designed or too restrictive can hinder the progress of the industry and have unintended effects (Tertoolen et al. 1998). There is a growing need for data-driven assessments of policy measures, particularly regarding their effect on climate patterns and issues such as pollution (Gössling et al. 2015). Transportation policies are now prominent topics in global discussions, as they play a crucial role in addressing environmental concerns. Stakeholders in various sectors recognize the importance of these policies in reducing the ecological footprint of the industry and promoting sustainability (Oberhofer and Dieplinger 2014).

The existing research on cost overruns in projects has mainly focused on project execution, but there is a gap in understanding the entire project life cycle due to limited access to comprehensive data (Flyvbjerg et al. 2006). Obtaining high-frequency data is crucial in gaining a comprehensive view of project dynamics from start to finish (Bianchini Ciampoli et al. 2020). The scarcity of data is mainly due to limited sources of information. Unlike other industries, the services sector has traditionally reported turnover data on a quarterly basis, but there is a shift towards monthly reporting starting in 2021, as required by the EU regulation (European Union 2019) entering in force from 2024. Therefore, there is still an incomplete availability of monthly services turnover data during the transition period, considering also the derogations applied to some countries.

The current work addresses, through the sparse temporal disaggregation approach, the previously mentioned gap investigating the possibility of using different data sources to quickly estimate phenomena of interest when there is a scarcity of the relevant data.

3 The dataset

The Italian turnover index 2015 = 100 is released monthly for industry and quarterly for market services by ISTAT and is crucial for the estimation of the components of the GDP from the production side. The delay in publication is around 55 days from the end of the reference period, even if recently a preliminary version of the turnover in industry was made available after 30 days for the needs of the quarterly national accounts and internally to ISTAT only. The index of turnover in transport, our target series, is therefore only quarterly and is not available for the last quarter on the occasion of the flash estimation. The full set of 21 releases from 2016:Q4 to 2021:Q4 are shown in Fig. 1 for both the unadjusted (upper panel) and seasonally adjusted series (lower panel) over the sample 2010:Q1–2021:Q4. The official seasonally adjusted series are obtained using TRAMO-SEATS whose model specifications are available upon request.

Note that, in contrast to the unadjusted data, the seasonally adjusted series undergo a more frequent revision, which consists of two essential components: the update strategy and the revision period. The update strategy governs the modification of the options and models for the seasonal adjustment as new observations are made or past observations are revised. These adjustments are easily discernible by comparing the final sections of the two plots.

Moreover, the revision period has great importance for determining the precise number of periods subject to revision and subsequent public availability upon the release of new series. It is important to note that the unadjusted series also undergo certain adjustments, albeit of smaller magnitude. This is exemplified by the conspicuous change of base that occurred in 2015 for both series.

Fig. 1
figure 1

Italian quarterly turnover index in transport: comparison between unadjusted (upper panel) and seasonally adjusted series (lower panel) for 21 releases from 2016:Q4 to 2021:Q4

Full size image

For this reason, it is extremely important to produce an accurate nowcast of turnover in transport and, possibly, to estimate it at a monthly frequency. For this purpose, three potential sets of predictors observed at a monthly frequency over the period 2010:1–2021:12 are considered, all available at \(t+30\) days:

  • Monthly business survey in transport (denoted hereafter by BS) which provides valuable insights into managers’ assessments of key economic variables within their companies. In more detail, the monthly questions refer to current assessments and expectations concerning the main activities of service firms (business market development, evolution of the demand and of employment over the last three months, as well as expectations of demand, the economic situation, employment, and prices). The data are available at the following link: Monthly Business Survey and their list is provided in Table 13 of the Appendix A together with the plot of the levels in Figs. 7 and 8. The reporting units are generally asked to select between three options, namely a positive, neutral, or negative answer. Their answers are then elaborated in the form of multiple percentages and afterwards also aggregated in the form of ‘balances’, which are constructed as the difference between the percentages of respondents giving positive and negative replies. In turn, the balance series are used to build composite indicators. For each surveyed economic activity, confidence indicators are calculated as the arithmetic means of the answers to a selection of questions closely related to the reference variable. For the market services companies, the calculation of the confidence climate includes the assessments as well as the expectations regarding the order books and the assessments of the business trend.

  • Data from Assaeroporti (The Association of Italian Airports, representing 28 airport management companies for 37 Italian airports) referring to the number of commercial, cargo and passenger movements in both national and international flights (AA). In more detail, the monthly traffic data provided by each airport management company show: the total number of arriving/departing aircraft, the total number of arriving/departing passengers, including direct transits (i.e. passengers transiting through an airport and departing using an aircraft with the same flight number as the arrival), the total amount in tons of traffic and incoming/outgoing mail. In our dataset we include 18 aggregate series, detailed in the Appendix A and available at Assoaerporti. Their list is provided in Table 14 of the Appendix A together with the plot of the levels in Fig. 9.

  • Industrial turnover Indexes (denoted hereafter by IT) measuring the monthly evolution of sales by industrial enterprises at current prices. In our dataset, we consider the total/domestic/foreign turnover for the main industrial groups (MIGS): consumer durables, consumer non-durables, intermediate goods, and energy, as well as 14 manufacturing subsections of economic activity, as explained in detail in Appendix A. The data are available at Turnover in Industry and their list is provided in Table 15 of the Appendix A together with a plot of the levels in Figs. 10, 11 and 12. As explained, this subset has recently been made available at 30-days for the round of GDP flash estimation only, but for both simplicity and uniformity with the other data we assume that it has been the same over the entire period treated by the real-time forecasting exercise.

A seasonal adjustment exercise was conducted using the TRAMO-SEATS version of the JDemetra+ 2.2.2 program on the complete set of potential predictors over the full sample period 2010:Q1–2021:Q4. The main findings show that both the AA and IT sets are influenced by seasonality, while the BS set is not. Overall, the ARIMA model specification employs either the Airline or simpler forms over data in levels (there are only a few cases in which the data have been log-transformed). Calendar effects are observed in the majority of cases of IT but only in two cases of AA. Furthermore, an automatic model specification identifies, as expected, most of the outliers during the COVID period for almost all the series. Results are available upon request. The generation of 20 vintages for each predictor over the expanding window is accomplished using the seasonally adjusted model identified for the entire sample period.

Table 1 shows a summary view of the stationarity (according to the Dickey–Fuller test) and seasonal adjustment for all the series belonging to the three primary datasets.

Table 1 Seasonal and stationary time-series distribution in the full set of predictors
Full size table

4 The model

The temporal disaggregation problem we face in this paper can be set up as follows. We observe a low-frequency data stream of the Italian service turnover index, which is recorded quarterly, and we wish to estimate a monthly version of this that is temporally consistent during each quarter. We denote the observed quarterly series by \(\varvec{y}_q \in \mathbb {R}^{n_l}\). It contains \(n_l\) quarterly observations. The latent monthly version is denoted by \(\varvec{y}_m \in \mathbb {R}^{3n_l}\). To estimate \(\varvec{y}_m\), we make use of a collection of p monthly indicator series stored in \(\varvec{X}_m \in \mathbb {R}^{n \times p}\) coming from business surveys in transportation, the Assaeroporti data set and turnovers in the industry. In the setting where we have monthly observations of indicators outside of the quarterly sample period, i.e. \(n > 3n_l\), the last \(n-3n_l\) monthly periods of \(\varvec{y}_m\) can be extrapolated.

The well-established regression framework pioneered by Chow and Lin (1971), which is used by several NSIs to perform temporal disaggregation, assumes that the following relationship holds at the monthly level:

$$\begin{aligned} \varvec{y}_m = \varvec{X}_m \varvec{\beta }+ \varvec{u}_m \,, \end{aligned}$$
(1)

where \(\varvec{\beta }\in \mathbb {R}^p\) is the vector of regression coefficients to be estimated, and \(\varvec{u}_m\in \mathbb {R}^n\) is a residual vector. The residual vector \(\varvec{u}_m\) has zero mean and the covariance matrix \(\varvec{V}_m\). Chow and Lin (1971) assume the data generating process of \(\varvec{u}_m\) follows a first-order autoregressive process \(u_t = \rho u_{t-1} + \epsilon _t\) with \(\epsilon _t \sim N(0,\sigma ^2)\) and \(|\rho |<1\). This assumption of a stationary residual process allows a co-integrating relationship between \(\varvec{y}_m\) and \(\varvec{X}_m\) when they are non-stationary, a likely scenario with economic time series. Thus, the regression coefficient \(\varvec{\beta }\) measures both the long- and short-run effect of \(\varvec{X}_m\) on \(\varvec{y}_m\). The resulting covariance matrix \(\varvec{V}_q\) has the well-known Toeplitz form:

$$\begin{aligned} \varvec{V}_m = \frac{\sigma ^2}{1-\rho ^2} \begin{pmatrix} 1 &{}\quad \rho &{}\quad \dots &{}\quad \rho ^{m-1} \\ \rho &{}\quad 1 &{}\quad \dots &{}\quad \rho ^{m-2} \\ \vdots &{}\quad \vdots &{}\quad \ddots &{}\quad \vdots \\ \rho ^{m-1} &{}\quad \rho ^{m-2} &{} \quad \dots &{} 1 \end{pmatrix} \, , \end{aligned}$$

containing two unknown parameters \(\rho \) and \(\sigma ^2\) that must be estimated.

As the dependent variable \(\varvec{y}_m\) in (1) is not observed, the regression is pre-multiplied by the \(n_l \times n\) aggregation matrix \(\varvec{C}= \varvec{I}_{n_l} \otimes \varvec{1}_3\), which converts monthly series into their quarterly values to obtain the observable quarterly counterpart of (1) given by

$$\begin{aligned} \varvec{y}_q = \varvec{X}_q \varvec{\beta }+ \varvec{u}_q \,, \end{aligned}$$
(2)

where \(\varvec{y}_q = \varvec{C}\varvec{y}_m\), \(\varvec{X}_q = \varvec{C}\varvec{X}_m\) and \(\varvec{u}_q = \varvec{C}\varvec{u}_m\) with covariance matrix \(\varvec{V}_q = \varvec{C}\varvec{V}_m\varvec{C}^\top \). Equation (2) is now fully observed, and the parameters in \(\varvec{\beta }\) and \(\varvec{V}_q\) can be estimated.

The standard approach used by Chow and Lin (1971) and its extensions (Fernandez 1981; Litterman 1983) to estimate \(\varvec{\beta }\) in Equation (2) uses the feasible generalized least squares (FGLS) estimator:

$$\begin{aligned} \hat{\varvec{\beta }}&= \mathop {\mathrm {arg\,min}}\limits _{\varvec{\beta }\in \mathbb {R}^p} \bigg \{ \left\Vert \varvec{V}_q^{-1/2}(\varvec{y}_q - \varvec{X}_q\varvec{\beta })\right\Vert _2^2\bigg \} \nonumber \\&= (\varvec{X}_q^\top \varvec{V}_q^{-1}\varvec{X}_q)^{-1}\varvec{X}_q^\top \varvec{V}_q^{-1}\varvec{y}_q \, . \end{aligned}$$
(3)

The reason it is referred to as the feasible GLS estimator is because it is conditional on the parameters in \(\varvec{V}_q\), namely (\(\rho \), \(\sigma \)), which are unknown and therefore have to be first estimated by means of maximum likelihood. However, in our application we have \(p=143\) indicator series in \(\varvec{X}_q\) that we wish to model. In such a high-dimensional scenario the matrix \((\varvec{X}_q^\top \varvec{V}_q^{-1}\varvec{X}_q)\) becomes rank deficient and is no longer invertible; hence, we cannot find FGLS solutions. Therefore, we consider several options:

  • Applying the standard approach of Chow and Lin (1971) and its extensions by including just one variable in \(\varvec{X}_m\) in the model at a time or, alternatively, their mean.

  • An approach based on LASSO regularization to operate in a high-dimensional scenario \((p > n_l)\).

Focusing on the latter option, we choose to use the method of Mosley et al. (2022) who propose a regularized M-estimation framework for temporal disaggregation, which they call Sparse Temporal Disaggregation, whereby they find estimators of the form:

$$\begin{aligned} \hat{\varvec{\beta }} = \mathop {\mathrm {arg\,min}}\limits _{\varvec{\beta }\in \mathbb {R}^p} \left\{ \left\Vert \varvec{V}_q^{-1/2}(\varvec{y}_q - \varvec{X}_q\varvec{\beta })\right\Vert _2^2 + \lambda \Vert \varvec{\beta }\Vert _1 \right\} \, . \end{aligned}$$
(4)

The introduction of LASSO regularization incorporates a penalty term that encourages the regression coefficients of the indicator series, denoted by \(\beta _j\), to move close to zero, particularly when they contribute significantly to a large least-squares score in the feasible generalized least squares (FGLS) framework (Tibshirani 1996). This dual effect enables two key outcomes: first, it facilitates the identification of essential indicator series by promoting the shrinkage of less important coefficients, and second, it yields sparse estimates for the regression coefficients, enhancing the precision of the selected variables.

The solution paths, \(\hat{\varvec{\beta }}(\lambda ,\rho )\), conditional on a tuning parameter \(\lambda \) that controls the degree of shrinkage and the AR(1) residual parameter \(\rho \), are found using the least angle regression (LARS) algorithm of Efron et al. (2004). The unknown parameters \(\lambda \) and \(\rho \) are found by iteratively calculating and minimising the BIC score:

$$\begin{aligned} \left( \hat{\lambda }, \hat{\rho }\right) = \mathop {\mathrm {arg\,min}}\limits _{\lambda , \rho } \left\{ -2\mathcal {L}\left( \hat{\varvec{\beta }}(\lambda , \rho )\right) + \log (n_l) K_{\lambda ,\rho } \right\} \,, \end{aligned}$$
(5)

where \(\mathcal {L}\left( \hat{\varvec{\beta }}(\lambda , \rho )\right) \) is the log-likelihood of the regression (2) and \(K_{\lambda ,\rho }=|\{j: (\hat{\beta }(\lambda ,\rho ))_j \ne 0 \}|\) is the number of degrees of freedom. The bias in \(\hat{\varvec{\beta }}\) is reduced by re-fitting the selected support back into GLS. For full details of the algorithm, refer to Mosley et al. (2022).

Once an estimator for \(\varvec{\beta }\) has been provided, Chow and Lin (1971) show that the best linear unbiased solution for the latent monthly target series \(\varvec{y}_m\) is given by

$$\begin{aligned} \hat{\varvec{y}}_m = \varvec{X}_m \hat{\varvec{\beta }} + \varvec{V}_m\varvec{C}\varvec{V}_q^{-1}\big (\varvec{y}_q - \varvec{X}_q \hat{\varvec{\beta }}\big ) \,, \end{aligned}$$
(6)

where the first part \(\varvec{X}_m\hat{\varvec{\beta }}\) is the conditional expectation of \(\varvec{y}_m\) given \(\varvec{X}_m\) and the second part is an estimate of the monthly residual \(\varvec{u}_m\), obtained by disaggregating the observed quarterly residuals \(\varvec{y}_q-\varvec{X}_q\hat{\varvec{\beta }}\) to ensure temporal consistency between the estimates \(\hat{\varvec{y}}_m\) and observations \(\varvec{y}_q\).

5 Empirical application

The objective of our empirical application is to highlight the benefits of both monthly real-time forecasting and the temporal disaggregation of the quarterly turnover in the transportation sector using the information available at \(t+30\) days after the quarter and provided by a set of alternative indicators.

Notably, since the official turnover figures for the transport industry are typically released at \(t+55\) days, our objective is to illustrate how this approach supersedes the conventional practice of nowcasting the last unavailable observation using an ARIMA model.

The first strategy is to consider the extrapolation of the last observation coming directly from the temporal disaggregation regression which includes, as explanatory variable, one indicator at a time or their mean and choosing the best performer in terms of error statistics. Alternatively, we can use in the model a selection of indicators obtained through a sparse approach, as given in detail in Sect. 4.

The entire exercise has been conducted using the R environment, using the package ‘RJDemetra’ for the ARIMA exercise, and both ‘DisaggregateTS’ and ‘LARS’ for the LASSO application.

The forecasting exercise moves in a pseudo-real-time context using the latest available data on the cut-off quarter 2017:Q1 until the last quarter considered in this exercise, which is 2021:Q4. Therefore, in all, we consider 21 vintages of disaggregations for each model specification. We opt for the use of a recursive window, letting all the available data be part of the in-sample regression whose parameters are used to generate the forecasts, in order to have more stable estimates given the low number of available observations. At each step of the real-time experiments considered hereafter, the parameters of the model are always re-estimated.

Upon each release of the monthly disaggregated transport index, we also obtain extrapolated estimates for three months into the future, through the set of selected alternative indicators. At the first round, the data are about the quarterly transport index over the quarters 2010:Q1–2016:Q4 and the alternative indicators are about the months January 2010–March 2017. This process repeats sequentially until the 20th iteration, where the transport index covers the quarters 2010:Q1–2021:Q3 and the alternative indicators span from January 2010 to December 2021. A 21st iteration is also computed, encompassing the whole of the sample period 2010:Q1–2021:Q4, serving for both comparative and diagnostic purposes.

In reference to the sparse approach, the nowcasts are generated by a newly specified model before each forecast round. This adaptive modelling approach allows us to complement the existing evidence on the relative contribution of different variable types to the nowcasts of transport turnover.

In fact, a further aim of the exercise is to make a selection of alternative indicators stable enough for a suitable horizon and to evaluate their forecasting performance with respect to other standard methods.

We use information at three different levels of aggregation: (i) singly; (ii) applying the simple mean; (iii) selecting dynamically the indicators with LASSO. We replicate our exercise on both unadjusted and seasonally adjusted series.

5.1 Accuracy measures

For the purpose of comparing the nowcasting performance of the various models, we compute measures of the accuracy of the forecast and in particular to the mean absolute error (MAE), which is calculated as follows:

$$\begin{aligned} MAE = \frac{\sum _{\tau =n_l-19}^{n_l}|Y_{\tau }-\hat{Y}_{\tau }|}{20} \end{aligned}$$
(7)

where 20 is the number of observations of the forecast sample, i.e. the quarters from 2017:Q1 to 2021:Q4 and where \(\hat{Y}_{\tau }\) denotes the one-quarter ahead forecast of the observed turnover index \(Y_{\tau }\) over the last 20 quarters \(\tau \) for \(\tau = n_l -19, \dots , n_l\). \(\hat{Y}_{\tau }\) is obtained as the mean of three monthly extrapolated values at quarter \(\tau \).

Adhering to the best practices employed by NSIs, our main benchmark is given by the ARIMA model, whose MAE is 4.652 for unadjusted data and 4.204 for seasonally adjusted data.

Similar statistics, but referred to the revision of each vintage to the next one, are computed at a monthly frequency. We refer to them as mean absolute revision error (RMAE).

Write \(\hat{y}_{\tau ,j}, j = 1, 2, 3\) for the estimates of the monthly turnover in transport conditional on the quarterly total of quarter \( \tau - 1\), and write \(\tilde{y}_{\tau ,j}, \hspace{0.2cm}\) for those conditional also on the quarterly total of the quarter \(\tau \). Their difference \(\hat{y}_{\tau ,j} - \tilde{y}_{\tau ,j}\) is the revision due to the accrual of the information on the quarterly total for three horizons (one, two and three months), in such a way that

$$\begin{aligned} RMAE_{j} = \left[ \frac{1}{20} \cdot \sum _{\tau =n_l-19}^{n_l} \left| \hat{y}_{\tau ,j} - \tilde{y}_{\tau ,j}\right| \right] , j=1,2,3. \end{aligned}$$
(8)

5.2 Single index regressions

In this section, we provide a comprehensive comparison of two of the most popular temporal disaggregation approaches adopted by NSIs: the Chow–Lin model and the Fernandez model. These two methods are applied \(143+1\) times by a simple specification of the turnover index over the constant term and a single standardized indicator at a time, where the last one concerns the mean computed over the full set of standardized indicators. The data are unadjusted and the exercise is made in real-time using the 20 recursive windows explained in Sect. 5.

For each method, we calculated the MAE and the estimated log-likelihood value related to the 21st step, computed as suggested by Proietti (2006) Equation (14).

These metrics serve to measure the accuracy and reliability of each approach and to gain a deeper understanding of their respective strengths and weaknesses. Our findings reveal that the Fernandez method slightly outperforms the Chow–Lin method.

Table 2 Single index results over unadjusted data
Full size table

Table 2 presents the model ranking, the variable name (details in Appendix A), its reference dataset, the MAE (the best performer in boldface), the log-likelihood value for each single indicator model applying the Fernandez and Chow–Lin method. Mean refers to the mean of all the 143 standardized indicators. The ranking is based on the MAE and the list is driven by the Fernandez approach. The likelihood refers to the estimation over the full sample.

Comparing the top 10 and bottom 5 indicators relative to the Fernandez model, our results suggest that Mean is the most reliable. The same occurs applying the Chow–Lin method. Indeed, it achieves a MAE of 2.26 for the former and 2.34 for the latter.

On the other hand, the poorest result is obtained, for the Fernandez approach, by the expectations of the demand over the next three months decreased (bh49_p) obtaining a MAE of 7.29 and by the foreign turnover of textiles, apparel and leather (fie_CB) for the Chow–Lin method whose MAE is 7.85. When comparing the two subsets, it is worth noting that 8 of the top 10 performers are common to both approaches, albeit in a different order. Furthermore, among the remaining 2 performers, it is intriguing to observe that avio cargo goods (air_cma), which ranks 9th for the Fernandez approach, is the \(54{\text {th}}\) performer for the Chow–Lin method.

Moreover, all the top 10 indicators perform better than the ARIMA benchmark. This result favours the use of information provided by alternative indicators in the forecasting process.

In terms of datasets, the largest part of the best performers belong to IT, followed by AA and BS. Interestingly, the first 4 positions, out of the Mean, all represent the energy sector (CD is for the NACE-code ‘coke and refined petroleum’). This is probably because transport is a sector highly dependent on energy in terms of cost and efficiency. Moreover, a large part of the economic shocks come from the energy sector recently affected by great inflation.

A similar real-time exercise is applied to the seasonally adjusted series. Their results appear in Table 3), where it becomes clear that, apart from the mean, the top 9 indicators belong to the AA dataset, according to the Fernandez approach. However, when using the Chow–Lin method, instead of the intra EU passenger (air_pue) and international passenger (air_pin) variables, we found that the variables of general aviation and other passengers (air_pav) and foreign turnover for intermediate goods (fie_int) variables are the most important.

These findings highlight the importance of using AA data to forecast seasonally adjusted data. Similarly to the case of unadjusted data, we observe that BS ranks last in terms of significance.

Finally, upon closer examination, it is intriguing to note that the MAE for seasonally adjusted data is globally lower compared to that of the unadjusted data. This result seems to come from the reduced variability of the seasonally adjusted series compared to the unadjusted series, even if the real-time nature of the empirical exercise would favour an opposite position, as seasonal adjustments themselves generate data revisions, as shown in Fig. 1.

Furthermore, the disparity between the two approaches (Fernandez and Chow–Lin) is not particularly significant. In fact, when comparing the most accurate predictor of each approach for two variables (air_mctt, air_mct), both representing distinct formulas for calculating the total commercial flight movements, we discover that the results are identical.

These findings emphasize the improved performance achieved through seasonal adjustments, enhancing the overall accuracy and reliability of our predictive models. However, it is important to acknowledge that the aim of the NSIs is often not focused solely on the seasonally adjusted series.

Table 3 Single index results over seasonally adjusted data
Full size table

Table 4 shows the parameter estimates of the two models with the full sample using only the constant and the best predictor selected with both datasets.

Table 4 Estimates of the regression coefficients
Full size table

5.3 LASSO analysis–sparse temporal disaggregation

A first exercise, under the LASSO approach, has been to run the sparse temporal disaggregation algorithm under the Chow–Lin model, letting all 143 potential predictors be selected at each step of the real-time experiment. Data are seasonally unadjusted and, as for the single index application, all the predictors are standardized. The model always includes the constant term. Concerning model parameters, we focus primarily on the \(\rho \) coefficient, which ranges from a maximum value of 0.99 in 2017:Q1 to a minimum of 0.13 in 2020:Q4, with last value of 0.70 released in 2021:Q4. The estimates of the \(\varvec{\beta }\) coefficients are not comparable, as the selected variables included in the model change over time.

The result has been that just 71 of the 143 indicators appear almost once in the selection set over the 21 estimation windows related to the quarterly time span of the target. A view of this subset is provided in the heatmap presented in Fig. 2, where the y-axis represents the date observed from 2016:Q4 to 2021:Q4 and the x-axis represents the index of the indicators out of the 71 selected almost once in the exercise. The colours indicate membership dataset: Business Surveys (BS) in red, Assaeroporti (AA) yellow, and Industrial Turnover (IT) orange. Table 5 provides the subset of indicators selected almost 5 times.

Fig. 2
figure 2

Chart of the indicators selected at each extension of the sample window. The y-axis displays the date observed from 2016:Q4 up to 2021:Q4. Refer to Table 5 for the main subset of indicators indexed on the x-axis. The colours indicate the membership dataset: red = Business Surveys (BS), yellow = Assaeroporti (AA), orange = Industrial Turnover (IT)

Full size image

At first glance, in terms of the distributions of the sources of data, Fig. 2 shows that the 78% of AA has been selected together with 63% of BS and just 32% of IT. This means that with respect to the original distribution, AA and BS are more represented, to the detriment of IT. However, the presence in the selected set is stable only for a few indicators belonging to the three datasets. Some others seem representative of specific events or shocks. This is especially true for BS indicators, of which around 50% were selected less than 3 times up to 21.

The dynamic selection moves in a range that goes from a minimum of 10 indicators chosen in 2016:Q4 to a maximum of 24 indicators in 2021:Q2 and 2021:Q3. The 10 indicators selected at the first forecast round (sample up to 2016:Q4) include: sh_a (balance of business situation development over the past three months), nh_d (answers to the question of the evolution of the demand over the past three months marked as ‘remained unchanged’), ah_g (evolution of the employment over the past three months as ‘increased’), nh49_g (evolution of the employment over the past three months land transport and transport via pipelines as ‘remained unchanged’) from BS, fit_CA (export turnover in food, beverage and tobacco), fie_B (export turnover in mining and quarrying), fid_CB (domestic turnover in textiles...), fie_CI (export turnover in computers and electronics) and fie_CK (export turnover in machinery and equipment) from IT and only air_ptt (total passengers) from AA.

While the 24 indicators selected in the forecast run related to 2021:Q2 and 2021:Q3, the maximum number of selected indicators, include only two of the series selected at the beginning (air_ppt, fit_CA) and a largest amount of BS series to the detriment of AA and IT. This outcome can be attributed to the ability of the BS series to capture temporary changes during the post-COVID period. Interestingly, both estimation windows in 2021:Q2 and 2021:Q3 share approximately 50% of the same series.

Table 5 Selected indicators and their frequency across the 21 sparse disaggregations
Full size table

From an economic point of view, the table shows the absence of a dominant significant predictor for the transport sector in our set of data. This result is not in line with the single indicator approach, which found the energy sector to be predominant. Interestingly, while the variables belonging to the AA dataset are more or less the same, during the crisis, the inclusions of some new variables let the IT spectrum change, while for the BS series, we see a sparse structure involving a larger number of series, above all during the COVID crisis.

5.3.1 Fixing the number of variables selected in sparse temporal disaggregation

To keep the number of selected indicators stable in each nowcasting period, we adjust the sparse temporal disaggregation algorithm to produce \(\varvec{\beta }\) estimates of a desired support.

We set the desired level of support to a specific number, which falls within the range between 1 and 24, the maximum number of indicators selected during the free selection exercise represented in the heatmap of Fig. 2.

The sparse temporal disaggregation method of Mosley et al. (2022) uses the LARS algorithm to find \(\varvec{\beta }\) solution paths. At each step of the algorithm, for a given value of the tuning parameter \(\lambda \), an indicator enters or leaves the selected support set and, therefore, there may be multiple values of \(\lambda \) that lead to the same support size in \(\hat{\varvec{\beta }}\). Mosley et al. (2022) then find the best \(\lambda \), and therefore the best support size, by minimizing the BIC. We propose to adjust this by collecting all values of \(\lambda \) in the LARS algorithm that generate a solution of the desired support (between 1 and 24) and minimize the BIC over these \(\lambda \) values. This gives the best coefficient estimate of fixed size for that particular nowcasting window.

We find that the number of selected indicators varies within the range between 5 and 82 if we fix the number of variables to be selected at each time equal to 1 or 24, respectively. Table 6 shows the MAE under the Fernandez model applied to a fixed number of free selected indicators.

The Table shows that the best outcome is achieved when a fixed number of seven indicators is selected, despite never being chosen in the free exercise. Overall, the results strongly support the use of a moderate number of indicators. This is probably due to the fact that having too few indicators may result in the omission of crucial information, whereas having an excessive number can compromise the accuracy of the estimation due to an insufficient number of degrees of freedom.

It is crucial to recognize the importance of having an optimal number of indicators, as this ensures the inclusion of essential information while maintaining the integrity of the estimation process.

Table 6 Mean absolute error for models with a fixed number of variables selected at each point in time
Full size table

5.3.2 Fixing the set of variables selected in sparse temporal disaggregation

One of the primary issues associated with the release of real-time data from NSIs is related to the revision process. The main objective is to minimize the need for excessive revisions when new data are released. However, this objective may not be achieved when the set of parameters used to estimate the disaggregation model changes over time. To address this concern, we replicate the exercise by fixing the variable selection. In other words, we repeat the exercise in real time, keeping the variables consistent with the selection obtained at each estimation window of the free selection exercise of Fig. 2. The results are shown in Table 7.

Table 7 Mean absolute error for models with a fixed set of variables selected equal to those selected in the free exercise at each estimation window
Full size table

In contrast to the findings obtained through free selection, fixing the set of variables favours the inclusion of a greater number of variables in the explanatory set. In fact, when considering the MAE, the best result is achieved with the selection made in 2020:Q2, which includes 14 variables.

5.3.3 Robustness checks: seasonally adjusted data

The same exercise as those presented in Sects. 5.3.1 and 5.3.2 has been replicated over seasonally adjusted data. The results are shown in Tables 8, 9 and 10 and in Fig. 3.

The heatmap of Fig. 3 shows that the total number of indicators selected is equal to 62, lower than the 71 selected under the unadjusted data case of Fig. 2. Indeed, despite both cases demonstrating a substantial surge in the number of indicators, the seasonally adjusted data opt for a more limited selection of regressors, varying within the range between 6 and 23.

In terms of the distribution of the sources of data, the selection includes the 56% of AA, 46% of BS and just the 38% of IT. This means that on the one hand, this exercise preserves the same ranking as that of the unadjusted case, but on the other hand, both BS and AA see a significant reduction of their representation, in contrast to IT, which maintains a similar share as it had for the unadjusted case. Moreover, we see a great decrease in the number of BS variables.

Fig. 3
figure 3

Indicators selected at each extension of the sample window. The y-axis displays the date observed from 2016:Q4 up to 2021:Q4. Refer to Table 5 for the main subset of indicators indexed on the x-axis. The colours indicate the membership Dataset: red = Business Surveys (BS), yellow = Assaeroporti (AA) and orange = Industrial Turnover (IT)

Full size image

The dynamic selection chooses a minimum of 6 indicators in 2016:Q4 and a maximum of 23 indicators in 2021:Q4. The 6 indicators selected at the first forecast round (sample up to 2016:Q4) include: nh_d (answers to the question of the evolution of the demand over the past three months marked as ‘remained unchanged’) from BS, fie_CB (foreign turnover textiles, apparel and leather), fie_CG (foreign turnover plastics non-metallic), fie_int ( foreign turnover intermediate goods), fie_CI (export turnover in computers and electronics) from IT and air_mcnz (commercial flight movements) from AA.

Note that, also under unadjusted data, the lowest number is selected at the first estimation window 2016:Q4, while the largest number is selected in the post-COVID period.

Delving deeper into the analysis, Table 8 shows that, on average, each regressor is chosen less frequently, above all for BS and IT. Also in this case, more than before, it is not possible to find a dominant set of indicators.

Table 8 Selected indicators across the 21 sparse disaggregations for seasonally adjusted data
Full size table

Upon examining the fixed number results, Table 9 shows that seasonally adjusted data yield a lower MAE with the same number of regressors when compared with the corresponding Table 6 for unadjusted series. However, it is worth noting that even though a smaller number of indicators is selected in the case of seasonally adjusted data during the free selection exercise, the optimal outcome is achieved with a fixed number of indicators equal to 20. Conversely, the corresponding example under unadjusted data demonstrates its best performance with only 7 indicators. This discrepancy poses an issue about the quality of the informative content included in seasonally adjusted against unadjusted series, something which we will focus on later.

Table 9 Mean absolute error for models with a fixed number of variables selected at each point in time with seasonally adjusted data
Full size table

Finally, Table 10 presents the selection of indicators for the entire forecast period. It is interesting to note that the most effective indicators are from 2020 in both cases. Specifically, the selection for the second quarter of 2020 for unadjusted data and the selection for the third quarter of 2020 for seasonally adjusted data yield the best performance. This could possibly be attributed to the challenges associated with forecasting during the COVID period. As a result, the indicators chosen during this time appear to consistently outperform those selected from other time periods.

Table 10 Mean absolute error for models with a fixed set of variables selected equal to those selected in the free exercise at each estimation window with seasonally adjusted data
Full size table

Note that, although in absolute terms the best performer is the selection for 2020:Q3, Table 10 shows that an almost equal result comes if we consider the selection adopted in 2017:Q1. In order to compare the two strategies, we look at the forecasting error components provided by each of them. Figure 4 shows that if the selection for 2020:Q3, compared to that for 2017:Q1, reduces the forecast error during the COVID period, it seems to be a worse performer overall and particularly in the quarters of 2021.

Fig. 4
figure 4

Comparison of the errors for the selections of 2017:Q1 and 2020:Q3

Full size image

5.4 Forecasting results

In this last part of our empirical application, we provide a comparative view of all the best performers coming from both the single indicator and the sparse modelling approaches of the previous sections. For sparse models, we also consider the results obtained: in real time under the free selection of indicators; those obtained with the best performing selection according to a fixed number of indicators; and those obtained with best performers for both a specific time period and type of adjustment. The results of the sparse models are always related to the Fernandez specification, as it always performs slightly better than the Chow–Lin one.

The results are shown in Table 11 for both unadjusted data (upper section) and seasonally adjusted data (lower section). The model specifications go by rows, and the accuracy measures relative to the terms of RMAE\(_1\), RMAE\(_2\) and RMAE\(_3\) of Eq. 8 which look at the monthly horizons and to MAE at the quarterly frequency, by columns.

Table 11 RMAE and MAE of real-time monthly disaggregations for quarterly turnover index in transport
Full size table
Table 12 Estimates of regression coefficients of best performing sparse models with the full sample 2010:Q1–2021:Q4
Full size table

It is immediately seen that the best-performing models (in boldface) are those related to a specific time period, namely 2020:Q2 and 2020:Q3, respectively, for unadjusted and seasonally adjusted data. The details about the selected indicators and parameter estimates obtained under the full sample size are shown in Table 12. Note that the selection over unadjusted data includes 4 indicators from both Assaeroporti and the business surveys and 6 from industrial turnover for a total of 14 indicators, and that over seasonally adjusted data it includes 3 indicators from both Assaeroporti and the business surveys and 6 from industrial turnover for a total of 12 indicators. Furthermore, we ensure coherence with the algorithm’s selection by not eliminating in model estimation the non-significant indicators. These include 4 indicators for the unadjusted model specification and 2 for the seasonally adjusted case.

The results for the unadjusted (seasonally adjusted) data suggest that an accurate selection of the indicators allows an enormous improvement of the forecasting performance. Indeed, the free selection method of Sparse TD provides a reduction in MAE of around 38% (46%) with respect to ARIMA. However, both the best predictor and the Mean, both obtained according to the Fernandez single index regression, are in line with this result. By contrast, a clear advantage comes from the adoption of a selection strategy under the sparse TD approach with fixed regressors. Indeed, as shown in the bold lines of Table 11, this selection induces a further reduction in MAE of around 42% (40%) with respect to the free selection method and achieves a total gain of around 64% (68%) with respect to ARIMA and of 27% (24%) with respect to the best Fernandez model: the Mean for unadjusted and air_mctt for seasonal adjusted data.

Concerning the RMAE, as expected, it depends on the forecasting horizon, so that \(RMAE_1< RMAE_2 < RMAE_3\). Again, the best selection of Table 11 (in bold) outperforms all the competitors, above all over longer horizons.

The models of the last line of the two sections of Table 11 refer to the best selection for a given period relative to the opposite type of adjustment: that of 2020:Q3 resulting from the experiment of seasonally adjusted data applied to unadjusted data and that of 2020:Q2 resulting from the unadjusted data experiment over seasonally adjusted data. The purpose of this analysis is to evaluate whether the selection made for unadjusted data proves beneficial in enhancing the performance of seasonally adjusted data, or vice versa, in order to align the selection between the two datasets. According to these results, the selection for the seasonally adjusted data should be preferred.

5.5 Implications for distributed estimates

In Figures 5 and 6, we present the vintages of disaggregated unadjusted and seasonally adjusted estimates, respectively. In both figures, panel (a) presents the estimates related to the best performer according to the single index Fernandez model (the Mean for unadjusted and the air_mctt for seasonally adjusted data), while panel (b) presents those related to the best sparse TD approach.

From both figures, it emerges rather clearly that the estimates obtained through the single indicator appear both smoother and more stable across releases compared to the sparse TD approach. This is due to the stability of both the indicators included in the model (only one according to the singular indicator approach against 14 and 12 of the sparse approach for unadjusted and seasonally adjusted case) and the estimated model parameters during the forecasting exercise.

Fig. 5
figure 5

Vintages of monthly distributed estimates of the unadjusted turnover in transports using Mean as indicator (a) and the sparse TD approach (b)

Full size image
Fig. 6
figure 6

Vintages of monthly distributed estimates of seasonal adjusted turnover in transport using air_mctt as indicator (a) and the sparse TD approach (b)

Full size image

But the most questionable issue is the disaggregation of the unadjusted data, seen as a statistical artefact by Proietti and Moauro (2008) and the related literature. Indeed, the seasonal component at the higher frequency of observation is not observed for the quarterly target and in the models of our application it is implicitly extracted together with the nonseasonal component. This implies that the higher-frequency seasonal factors are proportional to those of the indicators, with a factor of proportionality given by the same regression coefficients which relate the target with the quarterly aggregation of the monthly indicators of Eq. 2. The conditions under which these hypotheses hold are very strong, imposing, on the one hand, the proportionality of unobserved seasonal components between the target and the indicators and, on the other hand, a set of restrictions on the covariance structure of the observed components, both seasonal and nonseasonal.

Some of these restrictions could be removed by moving to the class of structural time-series models. In short, we start from the basic structural model (BSM) which decomposes a monthly time series into trend + seasonal + irregular component and where seasonality is modeled by means of a combination of the 6 stochastic cycles associated to a monthly trigonometric form. These cycles correspond to the fundamental frequency of a twelve-month period, plus its five harmonics (corresponding to periods of 6 months, 4 months, 3 months, 2.4 months and 2 months). Disturbances of these models are normally and independently distributed with common variance. We refer once again to Proietti and Moauro (2008) Equation 2 for details. In other words, the statistical treatment of structural models allows the separate estimation of the 6 seasonal cycles related to the monthly time span, 2 of which are also observed at the quarterly frequency, namely those of 3 and 6 months.

According to Proietti and Moauro (2008), temporal disaggregation and seasonal adjustment are performed simultaneously using a multivariate BSM approach. When disaggregating quarterly data into monthly data, it is crucial to pay special attention to the structure of the covariance matrix of the seasonal component, determining the contemporaneous correlations between its disturbances. This matrix is generally assumed to be the same for all the harmonics, but under a mixed frequency set of data, this restriction becomes unfeasible, since some seasonal frequencies are observed and others are not. Assuming that the covariance matrix of the observed seasonal components are of full rank and have a nonzero covariance structure, that of the unobserved harmonics are of reduced rank because these components are not detectable. In other words, we can get a direct estimate of the harmonics only for the observed frequencies, but not for the others. For these last, we need to adopt a feasible restriction in order to obtain them indirectly. One way is to borrow the covariance structure of the observable harmonics for those of the unavailable ones according to a ‘common’ or a ‘similar’ factor restriction in the sense of Harvey (1989) Section 8.5 and Moauro and Savio (2005) Section 3.4. Such a restriction appears particularly appropriate in view of the fact that the standard trigonometric model for seasonality assumes that the different harmonics all have the same covariance matrix.

The recalled solution of common seasonals in some frequencies but not in others is an elegant way to relax the strongest assumptions of full proportionality among all the unobserved time-series components, but does not overcome the evidence that some seasonal frequencies are aliased over temporal aggregates. For this reason, we do not provide any example of this alternative, considering also that its implementation is complex and would involve a great effort from a computational point of view. Indeed, the state space forms tend to increase enormously in the multivariate setup for in the presence of the seasonal factors, as well as the number of unknown hyperparameters to estimate. Moreover, this would involve the adoption of approximate methods of estimation (e.g. the EM algorithm) that do not ensure convergence towards optimal estimates. Finally, international regulations (see, for example, Eurostat (2014) and Eurostat (2021)) impose to statistical agencies that a large amount of seasonal aggregates are officially delivered; therefore, simplicity is a crucial element for accomplishing this purpose.

6 Discussion

Our empirical analysis indicates that no single variable can comprehensively explain overall the turnover in the transport industry better than others. In more detail, our findings reveal that the industry turnover set outperforms others for unadjusted series, while the Assaeroporti series are better for seasonally adjusted data.

Focusing on the sparse approach, by adopting a free selection strategy such that the variables included at each step of the real-time experiment change over time, we can compare the two heatmaps presented in Figs. 2 and 3. These heatmaps indicate that although the seasonally adjusted series can be modeled with fewer variables at each point in time (horizontal view), a significant portion of the selected variables do not consistently belong to the set of predictors included in the main dataset (vertical view). This result is also proved by the quite equal number of total variables selected in the two cases.

Note that, the sparsity structure, which was initially observed only over the BS series for unadjusted data, is also evident in the selection of IT series when we move to seasonally adjusted data. By contrast, AA data present a quite stable structure in both cases. Interestingly, a greater number of series were selected during the COVID-19 period, probably in order to address the shocks caused by that crisis.

However, while this free selection approach may be the most suitable in terms of fit, it is not the most effective both in terms of forecasting error on quarterly observations and in terms of revision of the monthly disaggregation.

For this reason, allowing the set of variables to change over time can lead to instability in the distribution design, resulting in changes over time when transitioning from one release to another, as more data become available. This means that to ensure stability in the distribution design, it is crucial to carefully manage the changing set of variables over time.

Our robustness checks have indeed shown that fixing the set of indicators helps to reduce the component of forecasting error. In this regard, Fig. 4 illustrates that although the COVID pandemic significantly altered the volatility structure of the error component, maintaining a fixed set of variables guarantees, in any case, efficient results over time. For this purpose, it is worth noting that although forecasting and temporal disaggregation are closely related issues, forecasting is a simpler exercise than temporal disaggregation. Indeed, while forecasting involves estimating a limited number of future values, temporal disaggregation involves the distribution of values across the entire sample period, which implies the instability of estimates from one release to another. This is a feature of all indirect estimations which can be overcome only by direct observation of the phenomenon of interest at the desired frequency.

Finally, replicating the exercise using both unadjusted and seasonally adjusted real-time data, we are aware that when we break down unadjusted data over time, it becomes difficult to directly observe the seasonal component. This lack of direct observation often leads to speculation about its relationship with another indicator. However, by conducting this exercise with seasonally adjusted series, we are able to enhance the rigour of the process. It is important to acknowledge, though, that this approach simplifies reality to some extent, which can be seen as a limitation. This limitation poses a challenge when presenting unadjusted data.

7 Conclusions

The paper has discussed the results of the selection of a set of monthly indicators to be adopted as both predictors and reference indicators for distributing the quarterly index of the Italian transport turnover over a monthly time span. Based on a set of 143 indicators coming from three different sources, we employed a sparse temporal disaggregation approach extending the classical regression-based method of Chow and Lin, using a LASSO regularization framework that allows a high-dimensional number of regressors to be included in the model.

Our results show that the standard strategies adopted by NSIs and based on ARIMA models are outperformed by all the proposed alternative approaches. In more detail, it is sufficient to employ a well selected set of indicators to obtain a significantly improved forecasting performance. In particular, focusing on unadjusted data, the best performer is the simple mean of the full set of standardized indicators, yielding a reduction of 51% in terms of the forecast error for the turnover in transport with respect to the ARIMA model. A single regression approach that uses, as a regressor, the energy sector turnover yields a reduction in error of 35%. This second result, although less important in terms of gain in performance, appears particularly strategic as, unlike the first, it requires updating only one series, compared to all the 143 indicators necessary to compute the simple mean.

However, by implementing a sparse approach to select a fixed set of key variables, an additional improvement of 26% for unadjusted data (39% for seasonally adjusted data) is achieved. This is accomplished by including only 14 explanatory variables (12 for seasonally adjusted data).

In conclusion, the proposed approach surpasses the standard methods employed by NSIs, demonstrating the effectiveness of alternative strategies. By using a carefully chosen set of indicators, the forecasting performance can be greatly enhanced.

Moving to the context of temporal disaggregation, in the absence of a real proxy of turnover at the monthly frequency, the exercise of distributing the observed quarterly values into a monthly time series appears particularly challenging. Indeed, if, on the one hand, the choice of a single indicator is appealing since it provides stability of the estimates over different releases, on the other hand, it could lead to doubtful results since the real monthly pattern of transport turnover is uncertain. For this purpose, sparse TD takes advantage of averaging the monthly pattern of a larger set of indicators. By contrast, the real-time experiment reveals that using the sparse TD approach presents a risk of instability of distributed estimates over subsequent releases, a well-known issue connected with estimates using a multiple set of indicators.

Overall, we recommend the use of this procedure for a careful selection of indicators in the context of temporal disaggregation, which constitutes a useful addition to the available menu of procedures for official statistics.