The implementation and operation of AI applications can involve considerable risks. From a regulatory perspective, the ICT risks inherent in AI systems demand particular attention. AI systems are assessed in the same way as general ICT systems in terms of their risk profile, complexity and the functions they support, such as processing sensitive data or supporting critical or important functions.
Many AI systems cannot be operated without the use of cloud services. This is why ICT third- party risk is also very important. The strategic issue of dependence on (a small number of) providers is not the only factor to consider here.
Furthermore, ensuring data quality and integrity can represent a major challenge. Manipulated or corrupted data can significantly impair model performance and security. Equally, the uncontrolled reuse or unsafe disposal of AI applications can jeopardise sensitive model or company data.
Cybersecurity is also becoming increasingly important due to the use of complex AI systems. For example, vulnerabilities such as backdoors can be introduced into the model undetected during training. Furthermore, the insecure deployment of software (for example through open source libraries) can lead to attackers stealing or manipulating models. Likewise, adversarial attacks or unauthorised access during model use can lead to wrong decisions being taken by entities.
It is the responsibility of ICT risk management to manage these and other ICT risks. The following sections provide further details on selected DORA requirements for AI systems.