Model Inventory Rationalization

Having an inventory of “model” estimates is crucial in the efficient management of model risk. We refer to models here as estimates whose output is not known ex-ante and hence entail some uncertainty. Thus, estimates that are more qualitative in nature, e.g. projecting fees for the Debt Capital Market business of an investment bank, also fit into the “model” category as the output fees depend on what assumptions are considered in the projection. 

Model estimates need to be accurately identified, registered, and versioned to enable developers, validators and auditors to trace lineages and have a full overview of how these estimates evolve over time. By using an inventory that gathers info comprehensively, firms have better insights into what models are performing as intended, or are at risk of failing, and how significant they are for the organization. 

At the heart of any model inventory discussion, one needs to consider the effort in maintaining the inventory. In this context, the proportionality principle is widely accepted, and is typically the foundation of regulatory guidance on the matter. A firm that employs a relatively low number of models that are based on simple approaches and that do not have complicated interdependencies, may require a lower level of scrutiny on their models. On the other hand, a large bank that adopts the latest, advanced analytical techniques for decision-making is expected to put significant effort into managing and maintaining their models.

In practice, though, firms still face some key challenges that undermine their use of models and ability to manage model risk today. 

• Resource constraints
• Lack of in-house expertise
• Absence of standardized frameworks

These challenges often translate into the inability of a firm to clearly determine how a model inventory should be organized so that efforts are optimally allocated in accordance with the proportionality principle. 

So what are the factors firms have to take into account to optimize their model inventory? 

In order to decide whether an estimate is a model or not, one should have governance in place that allows model owners to declare and identify all estimates that are significant in the decision-making process.

Approaches to be used to identify models may involve both qualitative and quantitative analysis. Model owners are requested to fill in pre-defined questionnaires, at the end of which a conclusion is reached by construction. Other more quantitative types of analysis may instead involve the use of (ML) techniques to assess the uncertainty of the estimated outputs. 

A combination of the two approaches is usually recommended, as the former helps in answering the right questions and the latter (quantitative) helps to identify, in a more systematic fashion, potential model candidates that may be otherwise missed in a manual self-declaration process. 

Once models have been identified, the next factor to consider is what constitutes an entry in the inventory itself. Below we suggest a few alternatives that may be considered when making this choice. A model entry is:

• The umbrella model with many usages (e.g. the Heston model)
• Each usage of the umbrella model (e.g. Equity barrier options under Heston)
• A different implementation of a usage of the model (e.g. PDE model for Barrier Options under Heston)

In light of the proportionality principle referred to above, one often faces the trade-off between accuracy and pragmatism when this choice has to be made. This dichotomy typically relates to the analysis of the following drivers:

• Model complexity/transparency – e.g. if there are more than two levels of dependencies, it may be necessary to break down into multiple entries  
• Model risk appetite – ability for senior management (shareholders) to easily identify whether model risk concentration is in line with strategic views and risk tolerance
• Skills/resources – availability of required knowledge/headcount to appropriately manage model risk (e.g. do we have enough competent model validators?)
• Tiering – level of model risk is appropriately captured by the tier (e.g. a model may be Tier 1 if single entry. If split into two entries, though, each entry may be Tier 2)

Standardization offers firms many benefits. When correct standards are in place, financial firms can develop a sustainable competitive advantage by being able to enforce risk governance effectively across the different stages of the model lifecycle. Standardization also enables firms to implement automation in areas requiring manual and repetitive work, to minimize operational risks and to achieve efficiency gains. Thus, teams are able to work more efficiently and in a controlled manner.

Technology has taken a leap forward with the introduction of automation & standardisation mechanisms and workflow engines that address key challenges in model risk management on an enterprise level. Whether firms choose to build in-house solutions or implement a third-party application, today’s model inventory tools no longer function simply as central repositories where estimates are inventorized; they also serve as a platform where teams can efficiently manage tasks that are designed to take shape as a firm’s strategy and framework, risk appetite, requirements and model complexity evolve over time.

As regulatory requirements continue to change, and technology expands across different business practices, it is crucial for financial firms to adopt a forward-looking view on model inventories. 

More rigid legacy systems may appear to be satisfactory in the short term, for smaller firms in particular. However, as a firm’s model landscape evolves and becomes even marginally more articulated, legacy systems very soon become obsolete in terms of comprehensive and more aggregated model risk management. 

To effectively manage models (and non-models), firms have to step up and invest in a flexible tool that can: accommodate the needs of various stakeholders, determine the quantitative nature and uncertainty of outputs, control and monitor the applicability of underlying model assumptions and dependencies, standardize different business processes. A model inventory that is dynamic by design can take shape and evolve as firms mature and can help identify and control ongoing model change processes. 

More importantly, firms can increase the return on investment on their models as teams are able to streamline the whole model lifecycle more effortlessly. This allows them to focus on value-adding activities such as quantifying, understanding and managing model risk. 

Do you have questions about how a model inventory can work for you? Then please schedule a meeting with Efrem Bonfiglioli.