Do US consumer survey data help beat the random walk in forecasting mortgage rates?

In line with term structure theory, empirical studies suggest that it is difficult to beat the random walk in forecasting long-term interest rates. We ask whether consumer survey data on both mortgage interest rates and expected inflation help beat the random walk in forecasting the 30-year fixed rate mortgage. Using the vector autoregressive (VAR) modeling framework with the mortgage rate and consumer survey data as variables, we generate the mortgage rate forecasts for 1988–2016. Our forecast evaluation test results indicate that the VAR forecasts generally embody useful predictive information above and beyond that contained in the random walk forecasts for 2008–2016. The evidence is weaker for 1988–2007 in the sense that the VAR forecasts fail to outperform the random walk (but still contain distinct useful predictive information). In line with the notion that consumers are “economically” rational, our findings suggest that consumer survey data have become more informative due to the uncertainty created by the 2008 financial crisis. Subjects: Psychological Science; Economics; Finance


PUBLIC INTEREST STATEMENT
Long-term interest rates, including the 30-year fixed rate mortgage, are difficult to forecast. Both theory and empirical evidence suggest that the best forecast of the future rate is today's rate (a naïve forecast). Despite the inherent difficulty, market participants and policy-makers are regularly engaged in producing more accurate forecasts than the naïve benchmark. This study is another attempt at providing accurate forecasts of mortgage rates for 1988-2016, using consumer survey data on both mortgage interest rates and expected inflation, derived from the long-running Michigan Survey of Consumers. Our results indicate that the consumer survey data are more helpful in producing accurate forecasts of mortgage rates for 2008-2016 than for 1988-2007. One explanation for this difference may be that the uncertainty created by the 2008 economic crisis has induced consumers to become more attentive by closely following relevant information and producing more informed assessments.

Introduction
Term structure theory suggests that long-term interest rates approximately follow a random walk (Pesando, 1979(Pesando, , 1980. This implies that such rates rapidly and fully reflect all relevant information so that future rate changes deviate from zero only in response to unexpected shocks. In line with this notion, empirical studies (including Baghestani, 2009;Brooks & Gray, 2004;Kolb & Stekler, 1996;Reichenstein & Shetty, 1989) show that professional forecasters fail to beat the random walk in forecasting long-term US Treasury bond rates.
Theoretically, in the absence of time-varying term premiums with the assumption that the term structure is determined according to the pure expectations theory, Reichenstein (2006, p. 117) shows that: where R m t is the long-term interest rate on an m-period bond at time t, and E t [R m t+1 ] is the long-term interest rate in t + 1 expected at time t. Similarly, r t is a one-period short-term interest rate at time t and r t+m is the short-term interest rate in t + m expected at time t. With m = 120 months for the 10year treasury rate, the right-hand side term in Equation (1) is nearly zero and the future rate is expected to be today's rate.
The mortgage industry uses the 10-year treasury rate as a benchmark in pricing fixed rate mortgages (Darrat, Dickens, & Al-Khazali, 2006). Accordingly, such rates often move in tandem with the 10-year treasury rate and thus display a random walk behavior (Baghestani, 2008a;Reichenstein, 1989). Following this argument, Baghestani (2008b) shows that the consensus forecasts of the US 30-year fixed rate mortgage from the Blue Chip panel of experts fail to outperform the random walk forecasts for 1992-2004.
In this study, we utilize consumer survey data on both mortgage interest rates and expected inflation (derived from the long-running survey conducted by the University of Michigan) and ask whether they help produce a more accurate forecast of the 30-year fixed rate mortgage than the random walk benchmark. Using the vector autoregressive (VAR) modeling framework with the mortgage rate and consumer survey data as variables, we generate the mortgage rate forecasts for 1988-2016. Our forecast evaluation test results indicate that these forecasts generally embody useful predictive information above and beyond that contained in the random walk forecasts for 2008-2016. The evidence is weaker for 1988-2007 in the sense that the VAR forecasts fail to outperform the random walk (but still contain distinct useful predictive information). In line with the notion that consumers are "economically" rational (Baghestani, 1992), our findings suggest that consumer survey data have become more informative due to the uncertainty created by the 2008 financial crisis.
The format of this study is as follows: Section 2 describes the data in addition to the VAR and random walk forecasts. Section 3 presents the forecast evaluation test results. Section 4 concludes.

Data and alternative forecasts
The data on the US 30-year fixed rate mortgage come from the Federal Reserve Bank of St. Louis database. The consumer survey data are derived from the University of Michigan Surveys of Consumers (MSC). Every month since January 1978, the MSC selects a nationally representative random sample of at least 500 US consumers to probe sentiment on personal finances, buying attitude, business conditions, and expectations. We focus on consumers' beliefs about both mortgage interest rates and expected inflation. In particular, one question asks, "Generally speaking, do you think now is a good time or a bad time to buy a house?" with the follow-up question "Why do you say so?" For the respondents who answer that it is a good time to buy a house, there are six options to select as reasons: Prices are low; prices will increase; interest rates are low; interest rates are rising (borrow in advance); it is a good investment; and times are good (prosperity). For the respondents who answer that it is a bad time to buy a house, there are five options to select as reasons: Prices are high; interest rates are high; it is a bad investment; there are uncertainties in the future (uncertain future); and I cannot afford it. For the purpose of this study, we utilize the number of respondents selecting the option that "interest rates are low" and the number of respondents selecting the option that "interest rates are high." 1 Baghestani, Kaya, and Kherfi (2013) find that the interest rate options have directional predictability for US home sales. In addition, Baghestani (2017) shows that consumer home buying attitudes can explain the behavior of US home sales in both tranquil and uncertain periods within 1978-2015.
As for expected inflation, the MSC asks: "During the next 12 months, do you think that prices in general will go up, or go down, or stay where they are now?" and "By about what percent do you expect prices to go up, on the average, during the next 12 months?" Using the individual responses, the MSC calculates and reports the consensus (mean) response as the measure of expected inflation. Ang, Bekaert, andWei (2007, p. 1206) note that "it is striking that the Michigan survey, which is conducted among relatively unsophisticated consumers, beats time-series, Phillips curve, and term structure forecasts." The importance of expected inflation follows the Fisher (1930) equation, which maintains that the nominal interest rate is the sum of the real interest rate and expected inflation. Baghestani (2010) shows that expected inflation calculated as the past rate of change in the GDP deflator over 10 years helps beat the random walk in forecasting the US 10-year treasury rate.
In this study, we formulate two VAR forecasting models. The first one includes three variables: the change in the mortgage rate, the change in expected inflation, and the number of survey respondents who select the option that "interest rates are low" and thus it is a good time to buy a house. For the January 1982-January 1987(1982.01-1987.01) sample period, the multi-equation Schwarz information criterion selects a VAR(2) model. That is, each of the three equations in the model includes a constant term and two lags of each variable on the right-hand side. We utilize the 1982. 01-1987.01 parameter estimates of the model to generate the forecasts of the mortgage rate for 1987. 02-198802- .01. The values for 198702- .02, 198702- .04, 198702- .07, and 1988.01 are, respectively, the 1-, 3-, 6-, and 12-month-ahead forecasts made at the end of 1987.01. Re-estimating the model for 1982.01-1987.02, we then use the updated parameter estimates to generate the forecasts of the mortgage rate for 1987. 03-198803- .02. The values for 198703- .03, 198703- .05, 198703- .08, and 1988.02 are, respectively, the 1-, 3-, 6-, and 12-month-ahead forecasts made at the end of 1987.02. This procedure is repeated until the last set of forecasts is generated for 2016.12-2017.11 using the 1982.01-2016.11 parameter estimates. The values for 2016.12, 2017.02, 2017.05, and 2017.11 are, respectively, the 1-, 3-, 6-, and 12-month-ahead forecasts made at the end of 2016.11. Accordingly, the sample periods for the 1-, 3-, 6-, and 12-month-ahead forecasts are, respectively, 1987.02-2016.12, 1987.04-2017.02, 1987.07-2017.05, and 1988.01-2017.11. For the sake of consistency, however, we focus on the 1988.01-2016.12 period for all forecast horizons. The second VAR model also includes three variables: the change in the mortgage rate, the change in expected inflation, and the number of survey respondents who select the option that "interest rates are high" and thus it is not a good time to buy a house. Again, for the 1982.01-1987.01 sample period, the multi-equation Schwarz information criterion selects a VAR(2) model. That is, each of the three equations in the model includes a constant term and two lags of each variable on the righthand side. We utilize the same procedure outlined above to generate the 1-, 3-, 6-, and 12-monthahead forecasts of the mortgage rate for 1988.01-2016.12. With the forecast horizon f (=1, 3, 6, and 12 months), the f-month-ahead VAR forecast of the mortgage rate (evaluated below) is a simple average of the two f-month-ahead forecasts obtained from the two VAR models described above.
As for the random walk forecasts, we set the 1-, 3-, 6-, and 12-month-ahead random walk forecasts equal to the actual mortgage rate for the last week of month t ending on Thursday. 2 This ensures that the random walk forecasts are comparable to the VAR forecasts, making our forecast evaluation test results valid. Finally, we evaluate the forecasts separately for the 1988.01-2007.12 and 2008.01-2016.12 sample periods. The latter period includes the recent financial crisis.

Forecast evaluation results
Our evaluation focuses on answering the following three questions: (1) Are the VAR forecasts more informative than the random walk forecasts?
(2) Are the mortgage rate forecasts unbiased?
(3) Are the mortgage rate forecasts directionally accurate? Figure 1 presents the forecasting timeline. As noted, A t , A t+1 , …, and A t+12 are the actual mortgage rates for the respective months. With the forecast horizon f (=1, 3, 6, and 12 months), PV t+f and R t+f are, respectively, the VAR and random walk forecasts of A t+f made at the end of month t. Figure 2 presents the time plot of the actual mortgage rate for 1988.01-2016.12. As can be seen for 1988. 01-2007.12 (2008.01-2016.12), the mortgage rate has a mean value of 7.69% (4.39%) with a high rate of 11.05 (6.48) and a low rate of 5.23 (3.35).

Figure 1. Timeline of the forecasts.
Notes: A t , A t+1 , …, and A t+12 are the actual mortgage rates for the respective months. With the forecast horizon f (=1, 3, 6, and 12), PV t+f is the VAR forecast of A t+f made at the end of month t. In addition, R t+f (=Aw t ) is the random walk forecast of A t+f , where Aw t is the actual mortgage rate for the last week of month t ending on Thursday.

Are the VAR forecasts more informative than the random walk forecasts?
To answer, we estimate the following encompassing test equation: where the actual mortgage rate in month t (A t ) is known at the time of the forecast. We subtract A t from both the actual (A t+f ) and forecast series to alleviate problems due to the non-stationarity of the series (Fair & Shiller, 1990). The VAR and random walk forecasts contain distinct predictive information when the estimates of γ 1 and γ 2 are both positive and significant. The VAR forecast contains more predictive information than the random walk when the estimate of γ 1 is positive and significant and the estimate of γ 2 is either insignificant or negative; the converse is also true. The VAR and random walk forecasts contain similar predictive information when the estimates of γ 1 and γ 2 are both insignificant. With the forecasts made in month t, the error term u t+f , while generally heteroscedastic, follows an (f − 1)-order moving-average process under the null hypothesis of rationality. In evaluating the 1-month-ahead forecast, we thus utilize the White (1980) procedure to obtain the standard errors corrected for heteroscedasticity. In evaluating the 3-, 6-, and 12-month-ahead forecasts, we utilize the Newey and West (1987) procedure to obtain the standard errors corrected for both the inherent serial correlation and heteroscedasticity.
Rows 1-4 of Table 1 report the OLS parameter estimates of Equation (2) along with the correct standard errors for 1988.01-2007.12. As can be seen in each row, the estimates of γ 1 and γ 2 are both positive and significant. This means that the 1-, 3-, 6-, and 12-month-ahead VAR and random walk forecasts contain distinct useful predictive information, and combining the two forecasts for each forecast horizon produces a forecast that is more informative than the individual forecasts (Granger & Ramanathan, 1984). To keep it simple, we generate the combined forecasts (denoted as PC t+f ) by taking a simple average of the VAR forecasts (PV t+f ) and the random walk forecasts (R t+f ) for 1988. 01-2007.12. 3 In order to see whether the combined forecasts are more informative than the random walk forecasts, we estimate the following encompassing test equation: where, again, A t is subtracted from both the actual and forecast series to alleviate problems due to the non-stationarity of the series. Table 2 reports the OLS parameter estimates of Equation (3) along with the correct standard errors. As can be seen, the estimates of δ 1 are positive and significant but Notes: A t+f is the actual mortgage rate in month t + f, and PV t+f and R t+f are, respectively, the VAR and random walk forecasts of A t+f made at the end of month t (the forecast horizon f = 1, 3, 6, and 12 months). The correct standard errors are in parentheses. R 2 is the adjusted R-squared. a Significance at the 10% or lower level. the estimates of δ 2 are all insignificant. Thus, the 1-, 3-, 6-, and 12-month-ahead combined forecasts for 1988.01-2007.12 are more informative than the random walk forecasts.
Rows 5-8 of Table 1 further report the OLS parameter estimates of Equation (2) along with the correct standard errors for 2008.01-2016.12. As can be seen in row 5, the estimate of γ 1 is insignificant but the estimate of γ 2 is positive and significant. This means that the 1-month-ahead random walk forecast is more informative than the corresponding VAR forecast. However, for the 3-, 6-, and 12-month-ahead forecasts in rows 6-8, the estimates of γ 1 are positive and significant but the estimates of γ 2 are all insignificant. This means that the 3-, 6-, and 12-month-ahead VAR forecasts embody useful predictive information above and beyond that contained in the random walk forecasts. Tables 1 and 2, in what follows, we focus on evaluating the combined forecasts for 1988.01-2007.12 and the VAR forecasts for 2008.01-2016.12. Table 3 reports the root mean squared forecast errors (RMSE) in addition to Theil's U coefficients. The U coefficient is the mean squared forecast error of the combined (VAR) forecast divided by the mean squared forecast error of the random walk forecast for 1988. 01-2007.12 (2008.01-2016.12). Consistent with the encompassing test results for 1988.01-2007.12, the 1-, 3-, 6-, and 12-month-ahead combined forecasts produce lower RMSEs than the corresponding random walk forecasts, with the U coefficients all below one. Consistent with the encompassing test results for 2008.01-2016.12, the same is true for only the 3-, 6-, and 12-month-ahead VAR forecasts.

Table 3. Relative forecast accuracy
Notes: See the notes for Tables 1 and 2. RMSE is the root mean squared forecast error, and U is Theil's U coefficient.
The U coefficient is the mean squared forecast error of PC t+f (PV t+f ) divided by the mean squared error of the random walk forecast for 1988. 01-2007.12 (2008.01-2016.12

Are the mortgage rate forecasts unbiased?
We examine unbiasedness by estimating: where P t+f is a general representative for the combined and VAR forecasts, and (A t+f − P t+f ) is the forecast error. The forecast is said to be unbiased if we cannot reject the individual null hypotheses that α = 0, β = 1, and α′ = 0 (Holden & Peel, 1990).
Rows 1-4 of Table 4 report the OLS parameter estimates of Equations (4) and (5) with the correct standard errors for 1988.01-2007.12. As can be seen, the 1-, 3-, 6-, and 12-month-ahead combined forecasts are all unbiased since we cannot reject the individual null hypotheses that α = 0, β = 1, and α′ = 0. Rows 5-8 report the OLS parameter estimates of Equations (4) and (5) with the correct standard errors for 2008.01-2016.12. As can be seen in rows 6-8, the 3-, 6-, and 12-month-ahead VAR forecasts are also unbiased since we cannot reject the individual null hypotheses that α = 0, β = 1, and α′ = 0. However, the 1-month-ahead forecast in row 5 fails to be unbiased since we reject the null hypothesis that β = 1. Despite the mixed results on unbiasedness, the null hypothesis that α′ = 0 cannot be rejected for any of the forecasts in rows 1-8. This means that the combined forecasts (for 1988.01-2007.12) and the VAR forecasts (for 2008.01-2016.12) are all free of systematic bias. In line with this conclusion, the size of the mean error (ME) estimates in rows 1-8, ranging from 0.008 to 0.086, are all small compared to the corresponding mean absolute error (MAE) estimates, ranging from 0.125 to 0.632.

Are the mortgage rate forecasts directionally accurate?
To investigate directional accuracy, we employ the procedure first introduced by Merton (1981) and Henriksson and Merton (1981) and later refined by Schnader and Stekler (1990), among others. We start with presenting the two-by-two contingency table in Table 5. As noted, the actual change is (A t+f − A t ) and the predicted change is (P t+f − A t ). The numbers of correct sign predictions are n 1 and n 2 , and the numbers of incorrect sign predictions are n 3 and n 4 . We use these statistics to calculate π All , π Up , and π Down . In particular, with n (=n 1 + n 2 + n 3 + n 4 ) defined as the sample size, π All = (n 1 + n 2 )/n is the overall directional accuracy rate; π Up = n 1 /(n 1 +n 3 ) is the proportion of correctly predicted upward moves; and π Down = n 2 /(n 2 +n 4 ) is the proportion of correctly predicted downward moves. (4)

Table 4. Unbiasedness test results
Notes: The correct standard errors are in parentheses. p-value is for testing the joint null hypothesis of unbiasedness (α = 0 and β = 1). ME is the mean forecast error, and MAE is the mean absolute forecast error. a Significance at the 10% or lower level. Rows 1-4 of Table 6 report these statistics for the combined forecasts for 1988.01-2007.12. In testing the null hypothesis of no (directional) association between the actual and predicted changes, we use Fisher's exact test and the χ 2 tests with and without Yate's continuity correction (Sinclair, Stekler, & Kitzinger, 2010). As indicated by superscript b, we reject the null hypothesis for only the 1-and 3-month-ahead combined forecasts, meaning that these forecasts are directionally accurate. For the 1-month-ahead combined forecast in row 1, π All = 0.71, π Up = 0.66, and π Down = 0.75. We employ the χ 2 test described in Berenson, Levine, and Rindskopf (1988, sec. 11.4.1) to test the null hypothesis that the proportion of incorrectly predicted upward moves (1 − π Up ) equals the proportion of incorrectly predicted downward moves (1 − π Down ). With the test p-value = 0.146 (> 0.10), reported in the last column of Table 6 (row 1), we cannot reject the null hypothesis and thus conclude that the 1-month-ahead combined forecast implies symmetric loss and is, therefore, of value to a user who assigns similar cost (loss) to incorrect downward and upward moves in the mortgage rate. For the 3-month-ahead combined forecast in row 2, π All = 0.59, π Up = 0.43, and π Down = 0.70. With the test pvalue < 0.001, reported in the last column of Table 6, we reject the null hypothesis of no asymmetry and thus conclude that the 3-month-ahead combined forecast implies asymmetric loss and is, therefore, of value to a user who assigns high (low) cost to incorrect downward (upward) moves in the mortgage rate.

Table 6. Directional accuracy test results
Notes: See the notes for Table 5. With n (=n 1 + n 2 + n 3 + n 4 ) defined as the sample size, π All = (n 1 + n 2 )/n is the overall directional accuracy rate; π Up = n 1 /(n 1 + n 3 ) is the proportion of correctly predicted upward moves; and π Down = n 2 /(n 2 + n 4 ) is the proportion of correctly predicted downward moves. In testing the null hypothesis of no (directional) association between the actual and predicted change, we utilize Fisher's exact test and the χ 2 tests with and without Yate's continuity correction. b p-values of these tests are all below 0.10.

Row No.
f Correct Incorrect π All π Up π Down p-value n 1 n 2 n 3 n 4 Forecast period: 1988Forecast period: .01-2007    Rows 5-8 of Table 6 report the statistics for the VAR forecast for 2008.01-2016.12. As indicated by superscript b, we reject the null hypothesis for only the 1-and 3-month-ahead VAR forecasts, meaning that these forecasts are directionally accurate. For the 1-month-ahead VAR forecast in row 5, π All = 0.62, π Up = 0.43, and π Down = 0.75, and for the 3-month-ahead VAR forecast in row 6, π All = 0.69, π Up = 0.29, and π Down = 0.90. With the test p-values < 0.001, reported in the last column of Table 6 (rows 5 and 6), we reject the null hypothesis of no asymmetry and thus conclude that the 1-and 3-month-ahead VAR forecasts imply asymmetric loss and are, therefore, of value to a user who assigns high (low) cost to incorrect downward (upward) moves in the mortgage rate.

Conclusions
Accurate forecasts of mortgage rates are important not only to housing market borrowers and lenders but also to policy-makers who are mindful of the strong link between the interest rate-sensitive housing market activity and business cycles (Bernanke, 2007). However, theory and empirical evidence suggest that long-term interest rates follow a random walk, meaning that the best near-term forecast of a long-term interest rate is today's rate. 4 In this study, we ask whether consumer survey data on both interest rates and expected inflation help beat the random walk in forecasting the 30year fixed rate mortgage. Numerous studies have investigated the predictive information content of consumer sentiment for macroeconomic indicators with mixed results. The findings of Carroll, Fuhrer, and Wilcox (1994) and Bram and Ludvigson (1998) suggest that consumer sentiment contains useful predictive information for consumer spending. Croushore (2005) replicates these studies with real-time data and shows otherwise. Batchelor and Dua (1998) find consumer sentiment useful only in predicting the 1991 recession. More recent studies by Dees and Brinca (2013), Christiansen, Eriksen, and Møller (2014), Kellstedt, Linn, and Hannah (2015), and Lahiri, Monokroussos, and Zhao (2016), however, present evidence in support of consumer sentiment as a reliable predictor of economic indicators.
In this study, we use the vector autoregressive (VAR) modeling framework with the 30-year fixed rate mortgage and consumer survey data on both mortgage interest rates and expected inflation to generate the forecasts of the mortgage rate for 1988-2016. Our test results indicate that these forecasts generally embody useful predictive information above and beyond that contained in the random walk forecasts for 2008-2016. The evidence is weaker for 1988-2007 in the sense that the VAR forecasts fail to outperform the random walk (but still contain distinct useful predictive information.) Such findings lend support to the notion that consumers are "economically" rational. In particular, Baghestani (1992, p. 282) points out that "To economically rational agents, the optimal forecast is that which balances the predictive benefit against the cost of gathering and processing information." Accordingly, it is reasonable to argue that the uncertainty created by the 2008 economic crisis has induced consumers to become more attentive by closely following relevant information and producing more informative responses to the survey questions. This argument is consistent with Lahiri et al. (2016Lahiri et al. ( , p. 1273) who show that consumer sentiment improves the accuracy of consumption forecasts (primarily through the services component). They further note that "the effect of sentiment is found to be stronger during the last recession when all categories of consumption seem to have been affected." Additional evidence is provided by Baghestani (2015) who uses the MSC data on both expected inflation and consumer sentiment to predict US gasoline prices for 2003-2014. He notes that the forecasts "are particularly more accurate for the period since 2008, reinforcing the notion that consumers are 'economically' rational" (p. 27).
Alternatively, one may argue that long-term interest rates have become easier to forecast due to the unusually low federal funds rates as a result of the Federal Reserve's quantitative easing (QE) in response to the 2008 financial crisis. Jarrow and Li (2014) show that the QE reduced the short-to medium-term (<12 years) forward rates but had almost no impact on longer term forward rates. Given that the mortgage industry uses the 10-year treasury rate as a benchmark in pricing the 30year fixed rate mortgage (Darrat et al., 2006), the question of whether the mortgage rate has become easier to forecast awaits subsequent research.