High-risk § 3.5.5

If an AI system intended to be used to evaluate the creditworthiness of natural persons or establish their credit score is also intended to be used for prudential purposes provided for by Union law to calculate credit institutions’ and insurance undertakings’ capital requirements, it will still be classified as high-risk under points 5(b) and (c) of Annex III. This follows from the clear specification in that provision that AI systems intended to be used for the use case listed therein shall only be exempted from high-risk classification if they are intended to be used for the purpose of detecting financial fraud.

The statement in Recital 58 AI Act that AI systems provided for by Union law for prudential purposes to calculate credit institutions’and insurance undertakings’ capital requirements should not be considered to be high-risk under the AI Act clarifies that AI systems used for those purposes fall outside the use case of point 5(b) of Annex III if they are not intended also to be used for the assessment of creditworthiness of and the establishment of a credit score for natural persons . An example of such a system falling outside the us e case of point 5(b) of Annex III is an AI system intended to be used by a financial institution to calculate solely the risk-weighted exposure of portfolios of loans to natural persons under the internal ratings-based approach (IRB) pursuant to Article 143 of Regulation (EU) No 575/2013, but the same system is not intended to be used for credit scoring or the evaluation of creditworthiness of natural persons. Such (normally) internal AI systems are not high-risk in an d of themselves as a result of the use test mentioned in Article 144(1)(b) of Regulation (EU) No 575/2013⁶⁵ and in Article 120 of the Solvency II Directive.

In this light, it should be recalled that it is not relevant wh at the main purpose of an AI system is for high-risk classification purposes . Provided one of the purposes of the system is the use case listed in point 5(b) of Annex III, the AI system should be classified as high-risk, regardless of the fact that the same AI system is also used for internal ratings and default and loss estimates under the IRB approach for the purpose of calculating own funds requirements.

If, however, financial institutions use different AI systems for both operations (e.g. a bank uses an AI system for credit-scoring that takes the IRB system as an input or uses an IRB system that takes the credit scoring as an input, but deviates in some elements from the IRB system), the AI system used for IRB purposes should not be classified as high-risk, since it is onlythe credit-scoring system that is intended to be used for the use case listed in point 5(b) of Annex III, AI Act. The same principles apply to insurance undertakings that widely use internal models and where those models play an important role in their system of governance in accordance with Article 120 of the Solvency II Directive.

Whether there are one or two underlying AI systems within the meaning of the use cases listed in Annex III requires a case-by-case assessment. Market surveillance authorities and providers must identify the underlying AI system(s) and their boundaries based on the definition of an AI system under Article 3(1) AI Act, drawing inspiration from the Guidelines on the definition of an artificial intelligence system⁶⁶ . This may require, inter alia, describing the hardware and software components of the AI system, as well as its objectives and its functionalities with a view to inputs and outputs, and its interface with other AI systems or its environment.

Where financial institutions, as a provider of a high-risk AI system falling within the use case of points 5(b) of Annex III, implement model changes of already approved IRB systems in a non-live production environment, this should not be considered as putting the AI system into service pursuant to Article 3(11) AI Act until approval is granted, since until then there is no intention, in practice, to deploy the AI system as implemented in the non-live environment.

In any case, high-risk AI systems placed on the market or put into service by financial institutions before the date of application of the high-risk rules are grandfathered and do not need to comply with the requirements and obligations , unless as from that date those systems are subject to significant changes in their design (see Article 111(2) AI Act and Section V below, as well as the relevant Union legislation on prudential requirements). For this purpose, the definition of material change to the credit-scoring within the IRB Approach as set out in the relevant Union legislation will be relevant in the context of the assessment of whether a high-risk AI system in point 5(b) of Annex III is subject to a significant change in design under Article 111(2) AI Act.