This chapter shows a way to, using simulation analysis, assess the performance of some of the most popular unit root and change in persistence tests. The authors do this by means of Monte Carlo simulations. The findings suggest that these tests show a lower than expected performance when dealing with some of the processes commonly believed to be found in the economic and financial data. The output signals that extreme care should be taken when trying to support a theory using real data. As the results show, a blind practitioner could get misleading implications almost surely. As an empirical exercise, the authors show that the considered test finds evidence of a unit root process in the US house price index. Nonetheless, as the simulation analysis shows, extreme caution should be taken when analyzing these results.
TopIntroduction
A distinctive property of the social sciences, economics in particular, is the impossibility to make controlled experiments where one could test the implications of a single variable movement unto the rest. Given these shortcomings, it is very important to develop trustworthy tools to assess if the scarce available data support a given theory. We cannot hope to repeat the environment's conditions as to test different hypothesis. When dealing with this kind of limitation, one powerful tool is regression analysis. This technique was pioneered by Galton (1886)1, and it has been widely used since then. However, some authors have pointed out that regression analysis has certain limitations such as potential confusion between correlation and causality. Used unwisely, this tool may produce misleading results. As Yule (1926) pointed out, it is possible to obtain nonsensical correlations such as the one between the “proportion of Church of England marriages to all marriages and the mortality rate over the period 1866-1911.”
In this context, simulation analysis has played a major role on assessing the properties of certain type of regressions and of the models used to fit the data. Probably some of the most famous applications of simulations are the ones due to W.S. Gosset on the Student-t distribution (Gosset, 1908a) and on parameter identification (Gosset, 1908b). Among the simulation techniques, one of the most recurred in statistics is the Monte Carlo method. The method, that owes its name to Meteopolis and Ulam (1949) in reference to the games of chance that take place in Monte Carlo, Monaco (the most well-known place for gambling at the time), consists in the evaluation of a deterministic model using random numbers as inputs. We can then compare the output to the one predicted by theory in the deterministic scenario to assess certain characteristics of interest for the researcher. We will use this kind of tools to determine the power of some of the most used tests for detecting unit roots and changes in persistence.
In an influential paper, Granger and Newbold (1974) argued that in applied econometric literature it was seen very frequently time series regression equations with an apparently high goodness of fit, as measured by the coefficient of multiple correlation R2. Moreover, these regressions typically display highly auto-correlated residuals, measured by a low value for the Durbin-Watson statistic. Performing Monte Carlo simulation of integrated processes of the ARIMA type, popularized by Box, Jenkins, and Reinsel (1970), they examined the potential for discovering spurious relationships, that is, to conclude wrongly that two or more independent series are related. The experiment provides strong evidence that the conventional significance tests are seriously biased towards rejection of the null hypothesis and, hence, acceptance of a spurious relationship. Since then, the topic attracted the attention of numerous researchers who showed different twists of this phenomenon. We can find spurious regressions under a lot of different specifications and data generating processes; see Ventosa-Santaulària (2009) for an excellent survey on this topic.