The prohibitions apply to any AI system, whether with an ‘intendedpurpose’33 or ‘general-purpose’ (i.e.that can serve a variety of purposes ) , for direct use or for integration in other AI systems.34 Accordingly, each operator should take measures for which they are best placed based on their role and control over the system in the value chain to ensure a responsible and safe provision and use of AI systems, balancing their risks and benefits with a view to achieving the twin objectives of the AI Act.
Deployers are thus expected not to use any AI system in a manner prohibited under Article 5 AI Act , including not to bypass any safety guardrails implemented by the providers of the system. While the harm often arises from the way the AI systems are used in practice, providers also have a responsibility not to place on the market or put into service AI system s, including general-purpose AI systems, that are reasonably likely to behave or be directly used in a manner prohibited by Article 5 AI Act.35 In this context, providers are also expected to take effective and verifiable measures to build in safeguards and prevent and mitigate such harmful behaviour and misuse to the extent they are reasonably foreseeable and the measures are feasible and proportionate depending on the specific AI system and circumstances of the case. In their contractual relationships with deployers (i.e. in the terms of use of the AI system), providers are also expected to exclude use of their AI system for prohibited practices and provide appropriate information in the instructions of use for deployers and regarding the necessary human oversight. For example, a general-purpose AI system used as a chatbot may deploy manipulative and deceptive techniques which is likely to cause significant harm . To prevent prohibited behaviour of the AI system and uses that are reasonably likely to manipulate, deceive and cause significant harms under Article 5(1) ( a) AI Act , the provider is expected to take appropriate and proportionate measures (e.g. , appropriate safe and ethical design, integration of technical and other safeguards, restrictions of use , transparency and user control, appropriate information in the instructions of use) before the AI system is placed on the market (Article 5(1) ( a) AI Act) to ensure the chat bot does not cause significant harm to users or other persons or groups of persons (see also section 3.2.3.(c)).
In certain cases, in particular where the prohibitions are linked to a very specific purpose of the system,36 providers may have limited possibilities to integrate other preventive and mitigating measures and will have to rely on primarily providing appropriate instructions and information to the deployers and the required human oversight and restricting prohibited use of the system. Where appropriate, such measures may also include monitoring for compliance with that restriction, depending on the means through which the AI system is supplied and the information at the provider’sdisposal for possible misuse . Any possible monitoring measures to detect misuse should not amount to a general monitoring of the activities of the deployers and should be in line with Union law. For example, a general-purpose AI system that can recognise or infer emotions should not be used by deployers at the workplaces or in education institutions , unless an exception for medical or safety reasons applies. However, the provider may not be in a position to know the specific context in which the emotion recognition functionality of the system will be used and whether an exception to the prohibition in Article 5(1)(f) AI Act may apply. Such providers may nevertheless explicitly exclude such prohibited use in their terms of use and include appropriate information in the instructions of use to guide deployers. They are also expected to take appropriate measures if they become aware that the system is misused for this specific prohibited purpose by specific deployers, for example, if such misuse is reported or the provider becomes otherwise aware, which may be the case if the system is directly operated through a platform under the control of the provider and the provider performs checks.