Applying the 3 Lines of Defense Framework to Model Risk Management

Models are fundamental to financial institutions. From evaluating an asset’s performance to forecasting the future value of investments, models act as a guiding light to sound business decisions. However, the increasing volume and complexity of transactions, coupled with the adoption of AI and ML for advanced analytics, has significantly increased the risk of these models producing unreliable and inaccurate outputs. 

And what is the cost of these erroneous models? Ask JP Morgan’s Central Investment Office (CIO), and they’ll tell you about the $6.2 billion in trading losses from a flawed Value at Risk (VaR) model and a further $900 million in regulatory fines. It is therefore vital for organizations to assess and manage model risk using a combination of comprehensive testing, robust governance policies, and independent reviews. One of the risk mitigation frameworks that blends all these strategies together is the 3 Lines of Defense (3LoD) Framework.

In this blog, we first learn what the 3LoD framework is and then apply it to model risk management, diving deeper into the roles and responsibilities of each line of defense.

The 3 Lines of Defense (3LoD) framework is an overarching risk governance framework that spreads out the responsibilities for operational risk management across 3 different functions. With clear roles and responsibilities assigned to each of these functions or “lines of defense,” organizations can ensure that they are well prepared to identify and address any dangers before they hamper operations. Let’s now look at what constitutes each line of defense for any organization:

• First line of defense: These are the business and process owners tasked with maintaining effective internal controls over operations on a day-to-day basis.
• Second line of defense: The second line of defense provides complementary expertise and support to the first line, but also formulates risk management practices. In an organization, these are typically managers with a risk management or compliance function.
• Third line of defense: The third line of defense consists of internal auditors who must provide independent and objective assurance to senior management and the board of governors that the company’s risk objectives are being achieved.

So, now that we know the basics of the 3LoD framework, let’s try applying it to mitigate model risk, which is ultimately a subset of operational risk.

When it comes to model risk specifically, there are several regulatory standards like the SR 11-7 and SS1/23 that lay out the best practices for model development, model implementation, model validation, governance, and other processes. Although these standards don’t constitute law but serve only a guidance, organizations can significantly reduce the risk of fines and penalties from regulatory bodies by staying compliant. Implementing the 3LoD framework is an effective way to stay compliant with these standards, and is a proven framework for reducing an organization’s exposure to model risk.

Who?

Model owners and model development teams form the first line of defense. They are generally domain experts with strong mathematical, statistical and programming knowledge (e.g., quants and data scientists) whose daily work includes preparing data, building and training models, as well as documenting development evidence.

Responsibilities

The first line of defense needs to ensure that appropriate risk controls are in place for model development and used to prevent and identify model risk at an early stage. These risk controls include:

• Model documentation – Model documentation includes the official set of documents and data that capture all essential information about a model over its lifecycle, i.e., from inception to use in production. It provides an inside view of the modeler’s thought process, including the rationale, assumptions, methodologies, tests, and derivations used in development. In addition to adding a layer of transparency, model documentation also provides a good foundation for governance measures such as model validation.
• Model implementation & testing – Models that have been conceptualized and developed from a more theoretical viewpoint need to be applied in practice to a functional system like a software product. Here, the team carries out rigorous testing procedures to examine the model’s behavior in different scenarios and identify any potential flaws. Model implementation & testing is particularly important for financial institutions since the results output by these models form the basis of future decision-making under different scenarios.
• Model monitoring – How do you assess whether a deployed model is working as expected? Model monitoring — continuous surveillance of a model’s performance — is the answer. It enables teams to quickly spot and address issues like emerging biases, and ensures that the models continue to deliver value and to be fit-for-purpose.
• Model maintenance – In the realm of finance, business conditions change and new regulations come into force fairly frequently. ML models that have been trained on historical data may not be factoring in these new aspects, which may lead to a reduction in performance and accuracy over time. Therefore, it’s crucial to have a mechanism in place that makes it possible to retrain models more frequently, and to provide them with clean, well-structured datasets so they can continue to operate effectively within the new business environment.

Who?

Model validation and model governance teams form the second line of defense and complement the first line in risk management.

Responsibilities

While the second line of defense can have certain overlaps in responsibilities with the first line as in model monitoring, some of the main tasks that the second line is uniquely responsible for are:

• Model validation – Initial model validation is the step before a model is deployed and used in production that makes sure the model will work well for its intended purpose. Here, the team assesses the accuracy, reliability, and performance of the model, often by testing it on independently sourced datasets that are different from the data used during development. This is very important since the ultimate goal of a model, and particularly ML models, is to apply the learning from its training data and work effectively with new scenarios it hasn’t seen before.
  Once a model is in production, model validation is repeated regularly with a frequency that depends on its level of model risk to guarantee that the model remains fit-for-purpose and conceptually sound.
• Model governance – Model governance ensures that models adhere to the risk policies and procedures within an organization over its lifecycle — from inception to retirement. Existing risk mitigation strategies in place are typically assessed for effectiveness. Best practices should be followed when developing and using the models. To achieve this, the model governance team establishes model documentation standards, formulates review and approval workflows, and discusses ethical considerations, among other things.

Who?

Internal auditors form the third line of defense. They generally perform model risk audits on a particular line of defense or type of model.

Responsibilities

Model risk internal auditors provide independent assurance on the functioning of the first two lines of defense to an organization’s governing body like its board of directors. They objectively assess the risk controls in place on models and check whether the compliance measures in place are being adhered to. The third line of defense serves as a support function to the governing body with impartial insights and helps in ironing out any inefficiencies in the end-to-end management of the model lifecycle.

Yields is an award-winning and adaptable technology that manages your end-to-end model lifecycle effectively. Its solution is pre-configured based on model risk management standards such as SR 11-7 and SS1/23, and can be further customized by business users to align with their internal model risk policies and procedures. It does not depend on internal IT teams for making changes.

A key benefit of the technology solution is that it facilitates collaboration across the 3 lines of defense. For instance, Yields offers automated model documentation capabilities, ensuring standardized documentation across your organization for streamlined model maintenance, governance, and robust validation processes. Furthermore, models can be tested at scale through standardized reusable routines with full reproducibility of results, improving the reliability and efficiency of your models.

If you’re interested in learning more about the Yields Model Risk Management platform while incorporating a 3 Lines of Defense framework for your organization, book a demo today!