How accurate are credit risk models in their predictions concerning Norwegian enterprises?

• Author: Syversten, Bjorne Dyre H.

Historically, banks' solvency problems are often due to losses on loans to enterprises. Credit risk associated with loans to enterprises is therefore an important aspect when Norges Bank assesses financial stability. Two different credit risk models are used in the analyses, Norges Bank's SEBRA model and the Moody's KMV Private Firm model. This article compares the quality of predictions made by the two models. The analysis shows that both models are good at selecting bankruptcy candidates among unlisted Norwegian enterprises and that the SEBRA model is somewhat better than the Moody's KMV Private Firm model.

1. Introduction

   There are clear methodological differences between the two credit risk models used by Norges Bank. The SEBRA model, which has been developed by Norges Bank, predicts bankruptcy probabilities on the basis of figures from the annual accounts of Norwegian limited companies. The Moody's KMV Private Firm model predicts the probability of default for large unlisted enterprises, based primarily on market information. SEBRA is thus an accounting-based model whereas the Moody's KMV Private Firm model may be characterised as a market-based model. This article compares the quality of the predictions made by these two models on the basis of predictions for Norwegian enterprises made after the financial years 1998-2001 and actual bankruptcies in the period 1998-2003.

   The structure of this article is as follows: Section 2 briefly presents the two models and comments on some methodological differences. Section 3 presents the data underlying the analysis, while Section 4 presents the results. Differences in the two models' treatment of different industries are discussed in Section 5, and a summary follows in Section 6.

2. Credit risk models

   2.1 Norges Bank's SEBRA model

   The SEBRA model predicts the risk of bankruptcy using 12 explanatory variables connected to figures from the annual accounts and some other enterprise characteristics. The model includes variables for earnings, liquidity, financial strength, industry, size and age. (1) The SEBRA model is based on a database containing annual accounts for all Norwegian limited companies. For the 2002 financial year, the database contains data concerning approximately 140 000 enterprises. The large majority of these enterprises are small. The SEBRA version of 2001 ("SEBRA 01"), which was estimated on the basis of annual accounts for the period 1990-1996, and the SEBRA version of 2003 ("SEBRA 03"), which was estimated on the basis of annual accounts for the period 1990-2000, were estimated on the basis of all enterprises in the database. A SEBRA version ("SEBRA Large") based on enterprises with annual turnover in excess of NOK 40 million was developed in connection with a previous comparison of SEBRA and KMV. The three SEBRA versions are fairly similar since there are only minor differences in the coefficient values of the various variables.

   The disadvantage of the SEBRA model is that new information comes in only once a year and that there is a time lag of nine months between the end of the financial year and the time most accounts are available in the database. For example, the bankruptcy predictions in June 2004 were based on annual accounts from 2002.

   2.2 The Moody's KMV Private Firm model

   The Moody's KMV Private Firm model, a model for unlisted enterprises, is an offshoot of the Moody's KMV Public Firm model, a model for listed enterprises. Whenever the models are discussed in the rest of this article, KMV is used as an abbreviation for Moody's KMV. The fundamental idea in the KMV Public Firm model is that an enterprise will default on its debt obligations if the market value of its assets becomes too low compared with the value of its debt. The level at which an enterprise is assumed to default on its debt obligations is called the default point. On the basis of studies of default statistics, KMV chooses to calculate this level as the value of the enterprise's short-term debt plus a portion of its long-term debt. The default point is thus assumed to be somewhat lower than the value of total debt. The calculation of the default point is based on information from the financial accounts concerning the enterprise's financial position. Market data are used to estimate the market value of the enterprise's assets. On the basis of the share price of the enterprise in question and the volatility of the share price, option pricing theory is used to estimate the market value of the enterprise's assets. A key variable in the KMV model is the distance to default, which is defined as the difference between the market value of the assets and the default point expressed in standard deviations. Using KMV's database of actual defaults, the distance to default is then converted to expected default probability (EDF). The greater the distance to default, the lower the expected default probability. As standard, the KMV model states the probability of default in the next 12 months for the enterprise in question. (2)

   Quoted share prices do not exist for unlisted enterprises. This means that the market value of an enterprise's assets must be determined in some other way. KMV's Private Firm model estimates the market value of an enterprise's assets as the enterprise's EBITDA (3) multiplied by a factor that is a function of share price movements for listed enterprises in the same industry, share price movements for listed enterprises in the same country and the size of the enterprise in question. The methodology used in the KMV Public Firm model is then used to calculate the expected default probability.

   One would expect the KMV Public Firm model, which is based on the market's continuous pricing of equity in each enterprise, to be more accurate in predicting default than the KMV Private Firm model. The drawback of the latter model is that the estimated market value of the enterprise's assets is based on average figures for somewhat similar enterprises and not on the market's continuous pricing of enterprise-specific risk factors. The SEBRA model predictions are compared with the predictions of the KMV Private Firm model because there are so few listed enterprises in Norway that it is not meaningful to make a comparison with the KMV Public Firm model.

   Moody's KMV has also developed an accounting-based credit risk model for unlisted enterprises called Moody's KMV RiskCalc. We have not tested SEBRA's predictions against this model since one important purpose of the test is to...