Does PRA SS1/23 cover LLMs?
SS1/23 applies to UK banks, building societies and PRA-designated investment firms with approval to use internal models for regulatory capital. Its definition of a model includes qualitative output, which the PRA said is meant to bring AI and machine learning with qualitative output into scope. Whether an LLM use is a model is for the firm to decide.
David McMillan, Governed AI Deployment. Checked against the sources on .
§01PRA SS1/23 para 1.2
Which firms it applies to
In the PRA’s words:
This SS is relevant to all regulated United Kingdom (UK)-incorporated banks, building societies and PRA-designated investment firms with internal model approval to calculate regulatory capital requirements. The expectations in this SS do not apply to firms which do not have permission to use internal models to calculate regulatory capital and third-country firms operating in the UK through a branch. However, the PRA considers that those firms may find the proposed principles useful, and are welcome to consider them to manage model risk within their firm.
PRA SS1/23 para 1.2
§02PRA SS1/23 Principle 1
What counts as a model
Firms should adopt this definition. In the PRA’s words:
A model is a quantitative method, system, or approach that applies statistical, economic, financial, or mathematical theories, techniques, and assumptions to process input data into output. The definition of a model includes input data that are quantitative and / or qualitative in nature or expert judgement-based, and output that are quantitative or qualitative.
PRA SS1/23 Principle 1
§03PRA PS6/23 para 2.23, PRA PS6/23 para 1.3
What the PRA said about AI
SS1/23 does not name AI or machine learning. The policy statement that published it, PS6/23, does. In the PRA’s words:
The PRA’s model definition intends to ensure that recommendation systems in client services and other AI/ML that deliver qualitative output are within the scope of the MRM policy. For example, machine learning models that uses data mining to seek to predict, narrow down, and find relevant content for users or recommend additional products to consumers.
PRA PS6/23 para 2.23
Its consultation had proposed managing the risks of AI in modelling techniques “to the extent that it applies to the use of models more generally”.
§04PRA SS1/23 Principle 1, PRA SS1/23 Principle 2, PRA SS1/23 Principle 3, PRA SS1/23 Principle 4, PRA SS1/23 Principle 5
What follows for a firm in scope
SS1/23 sets five principles: model identification and risk classification; governance; development, implementation and use; independent validation; and risk mitigants. A firm that decides an LLM use is a model brings it under all five. The controls matrix shows which controls the kit maps to each.
§05
Sources
- PRA SS1/23, Model risk management principles for banks (bankofengland.co.uk)
https://www.bankofengland.co.uk/-/media/boe/files/prudential-regulation/supervisory-statement/2023/ss123.pdf
- PRA PS6/23, Model risk management principles for banks (bankofengland.co.uk)
https://www.bankofengland.co.uk/prudential-regulation/publication/2023/may/model-risk-management-principles-for-banks
§06
Book a scoping call
Thirty minutes on one system: what it does, who is asking about it, and which engagement fits. Nothing to prepare. For firms in the UK and Europe.