What is Type-2 Error Rate

Early Warning Models (EWMs) are back on the policy agenda. In particular, accurate models are increasingly needed for financial stability and macro-prudential policy purposes. However, owing to the alleged poor out-of-sample performance of the first generation of EWMs developed in the 90's, the economic profession remains largely unconvinced about the ability of EWMs to play any important role in the prediction of future financial crises. The authors argue that a lot of progress has been made recently in the literature and that one key factor behind the prevailing skepticism relates to the basic evaluation metrics (e.g. the noise-to-signal ratio) traditionally used to evaluate the predictive performance of EWMs, and in turn to select benchmark models. This chapter provides an overview of methodologies (e.g. the (partial) Area Under the Receiver Operating Characteristic curve and the (standardized) Usefulness measure) better suitable for measuring the goodness of EWMs and constructing optimal monitoring tools.