When we use mathematical models to support decision making, we face model risk since mathematical models can produce incorrect outcomes. Models can fail due to technical mistakes, such as lack of mathematical rigor, data issues, implementation bugs, and other issues and these types of model risk incidents can be considered part of operational risk. However, models produce incorrect results as well simply as a consequence of the inherent uncertainty related to mathematical modelling. This in itself is a characteristic feature of modelling but a failure to recognize this uncertainty properly constitutes a key element of model risk. A final category of model risk accidents are related to using models that are not well understood, i.e. in situations when we are unable to properly explain why a model has come to a certain conclusion. In the latter case, there is a link with operational risk.
Model failure can lead to financial losses, regulatory or legal penalties, or damage to a bank’s reputation, resulting from the use of a model that contains data or assumptions that were not appropriate for the model.
In order to manage model risk, governance policies and frameworks should be applied. The corner stone of model risk governance is the implementation of three lines of defence:
- first line (the model developers) is responsible to persist in a structured document all assumptions, both on the data and on the actual model. Once a model is used in production, it is most often a first line's responsibility as well to monitor the model to detect issues as quickly as possible.
- second line (the model validators) has to independently review this document to ensure that it is self-contained, that the results are reproducible and that the limitations of the models are well understood by the decision makers, the users and the developers. This independent review consist of model validation.
- third line (audit) is responsible for verifying that the processes between first and second line are effectively implemented.
Two examples of model risk are the Credit Crisis of 2008 and the London Whale Trading Incident in 2012. In the Credit Crisis, banks were relying on very simple models to estimate mortgage risk. Although the used models were mathematically correct, they were too simple to deal with the highly complex derivatives that were traded at that time. In the London Whale Trading Incident, JPMorgan lost over $6 billion due to a mistake in a credit risk model. In both cases, the root cause was a lack of proper model risk governance that should have detected the flaws in the models that were being used.