The ECB’s latest update on TRIM outcomes shows that weaknesses in model validation generated the lion’s share of its findings in the area of counterparty credit risk. Ironing out these shortcomings and following best practices will be crucial to banks’ ability to manage model risks, and to avoiding any adverse impact on their decision-making processes.
The recent Targeted Review of Internal Models (TRIM) findings unveiled by the European Central Bank (ECB) related to model validation, together with the launch of the ECB annual validation reporting requirement, are two events that indicate growing supervisory focus on the model validation framework deployed by banks.
Model validation involves the processes and activities that verify models are performing as intended, and is a core element of model risk management (MRM). For instance, the Basel Committee’s minimum standards for internal ratings-based (IRB) institutions require a regular cycle of model validation “that includes monitoring of model performance and stability; review of model relationships; and testing of model outputs against outcomes.”1
Over the last few years supervisory expectations for model validation have evolved rapidly. November 2018 and July 2019 saw the ECB revised its Guide to Internal Models, which sets out a range of model validation requirements in both its General Topics and Risk-Specific chapters. Most recently, the ECB’s third update on TRIM outcomes published in November 2019 summarises the findings of on-site inspections (OSIs) focused on counterparty credit risk (CCR). Model validation not only generated more findings than any other CCR topic, but also the largest number of severe findings. In particular, the TRIM review “identified weaknesses in the scope and depth of the validation tasks and also shortcomings relating to back-testing due to inappropriate coverage, missing horizons, missing levels or risk measures (e.g. the exposure metric) or a lack of follow-up action.”2
With these findings in mind it may be worthwhile for banks to remind themselves of the ECB’s revisions to the CCR chapter of its Guide which, along with other topics, focused on model validation. Some of the key enhancements were:
Banks should also remember that these enhancements cover only a handful of the Guide’s full range of requirements. In addition to the above stated enhanced expectations, other important best practices include:
Given this background and the increasing reliance of banks on models for their decision-making, it is becoming critical for banks to establish a robust model validation framework that protects them from the consequences of misleading model outputs. In light of the ECB’s revised Guide to internal models and its TRIM CCR findings, we believe that banks should reinforce the following model validation measures, if they have not done so already:
Furthermore, the ECB’s supervisory priorities for 2020 – which include upcoming activities in relation to the European Banking Authority’s IRB repair programme - indicate that supervisory focus on the adequacy of internal models will continue to gain traction this year. It is therefore essential for banks to implement robust MRM programmes, including establishing strong model validation frameworks that can adequately determine models’ fitness for purpose.
To sum up, model validation is not only the subject of growing supervisory scrutiny. It is also vital to mitigating the risks of models generating unwarranted results, which could have significant adverse impacts on banks’ decision-making.