One of the key elements in effective Model Risk Management (MRM) is the development and maintenance of model documentation. Time-consuming, manual inputs are still dominant in model documentation, and forward-thinking financial institutions are looking for automation to improve the efficiency of the MRM function. In his recent presentation at the Advanced Model Risk USA conference in New York, Yields.io’s CEO Jos Gheerardyn addressed the recurring challenges financial services firms face in generating and maintaining model documentation. After his presentation, we had a chat with Jos about the topic and he stressed that without adequate documentation, model risk assessment and management will be ineffective. He also explained how technologies such as the Yields MRM Platform can help.
Why is model documentation important?
Model documentation includes a general description of the model and clarifies a model’s intended purpose, assumptions, limitations, and proper usage by banks. Documentation of model development and validation should be sufficiently detailed so that parties unfamiliar with a model can understand how the model operates.
Jos emphasized that maintaining up-to-date and comprehensive documentation promotes transparency and compliance, making it easier for regulatory bodies to grasp the decision-making process during model development. Effective documentation also streamlines model validation and maintenance, especially as model dependencies and complexity continue to grow. Jos explained, “Without proper model documentation, it’s difficult for stakeholders to comprehend the impact a specific model aims to achieve. Knowledge sharing is critical during the model lifecycle, especially as teams tend to change over time. Model documentation is a means of scalable communication.”
(This diagram illustrates the different stages in the model lifecycle, where each stage requires sharing of accurate documentation that can be challenging when done manually.)
What challenges do financial services firms face?
Jos identified several factors firms should consider as part of their model documentation process:
- Handovers across various stages within the model lifecycle necessitate constant documentation updates. Ensuring consistency is challenging when everyone has their own documentation style.
- Multiple stakeholders require information; without a comprehensive understanding of model documentation from different stages, it’s difficult for teams to perform their tasks effectively.
- Numerous dependencies within a model may affect the model itself, emphasizing the need for a systematic approach to content and documentation management.
- Model documentation is often manual, making it difficult for firms to systematically track and update documentation. Outdated documentation that no longer accurately represents the model can create operational risks.
Challenges in model documentation often arise from inconsistencies in the produced documentation, as well as varying quality, and outdated documentation. Inconsistencies in documentation stem from several factors. Often multiple documents per model have to be created: per geography (SR11-7 vs. CP6/22), and per stakeholder type (e.g., credit officer vs. internal model review team). Furthermore, model dependencies have to be captured accurately, which has become increasingly challenging with the rapid deployment and adoption of AI models.
The level of detail and quality of model documentation often varies significantly, particularly when outsourcing model development or validation activities. The author’s written language skills and the time they have available will also influence the quality of the documentation. Another important challenge is that model documentation is often outdated when content is not captured dynamically. This frequently happens when documentation is generated manually. An automated document refresh should be triggered, such as by model monitoring, recurring validation, model version upgrades, etc.
Last but not least, model documentation is a tedious, manual process.
Jos elaborated, “Models are continuously developed, monitored, and validated, requiring users to regularly update the model documentation. In practice, manual documentation can be labor-intensive, and people often overlook detailing all changes, leading to future problems.”
The Role of Technology
During his presentation at the Advanced Model Risk USA, Jos showcased how technology can help manage and streamline the model documentation process, enabling financial services firms to save costs and maintain compliance. He explained that 80% of elements found in the documentation are shared, allowing for standardization.
“By leveraging technologies such as the Yields MRM Platform, firms can manage individual content elements separately, addressing consistency issues. This standardization enables firms to automate processes, resulting in significant efficiency gains, which reduces the risk of non-compliance and ensures up-to-date model documentation whenever a model undergoes changes.”
Would you like to learn how a G-SIB leveraged the Yields MRM Platform to automate model testing and documentation? Explore our latest case here.
