Besides law enforcement authorities that are in principle the main deployers of AI crime predictive systems, the activities of private entities may also be covered by the prohibition in Article 5(1)(d) AI Act in some cases . That follows from the fact that, based on its wording, the prohibition does not apply exclusively to law enforcement authorities. Moreover, otherwise the prohibition might be easily circumvented, which would call into question its effectiveness.
That being so, the prohibition may be assumed to apply, in particula r, when private actors are entrusted by law to exercise public authority and public powers for the prevention, investigation, detection or prosecution of criminal offences or the execution of criminal penalties.139 Private actors may be also explicitly requested on a case-by-case basis to act on behalf of law enforcement authorities and carry out individual crime risk predictions. In those cases, the activities of those private actors could also fall within the scop e of the prohibition , if the applicable conditions are fulfilled and the exclusion does not apply. For example, a private company providing advanced AI-based crime analytic software may be asked by a law enforcement authority to analyse a large amount of data from multiple sources and databases , such as national registers, banking transactions, communication data, geo-spatial data, etc., to predict or assess the risk of individuals as potential offenders of human trafficking offences. If all the criteria for Article 5(1)(d) are met, such a use case could be prohibited.
Furthermore, the prohibition may apply to private entities assessing or predicting the risk of a person of committing a crime where this is objectively necessary for compliance with a legal obligation to which that private operator is subject to assess or predict the risk of persons committing specific criminal offences ( e.g., in case of anti-money laundering, terrorism financing). For example, a banking institution has an obligation under Union anti-money laundering legislation140 to screen and profile customers for money-laundering offences. If the bank uses an AI system to fulfil its obligations, that should be done based only on the data a s specified in that law which are objective and verifiable to ensure that the person s singled out as suspect are reasonably likely to commit anti-money laundering offences. The predictions must also be subject to human assessment and verification in compliance with that legislation141 in order to ensure the accuracy and appropriateness of such assessments. Compliance with that legislation will ensure that the use of individual crime prediction AI system for anti-money laundering purposes fall outside the scope of the prohibition in Article 5(1)(d) AI Act.
However, having regard to the focus on risk assessments relating specifically and exclusively to the commission of criminal offences that is evident from the wording of the prohibition as well as to the purpose of the prohibition as explained in Recital 42, if a private entity profiles customers for its regular business operations and safety or to protect its financial interests (e.g., detecting financial irregularities) without the purpose of assessing or predicting the risk of the customer committing a specific criminal offence, the activities of the private entities should not be considered to fall under the scope of the prohibition of Article 5(1)(d) AI Act.
In other words, in the absence of private parties having been entrusted by law certain specific law enforcement tasks, acting on behalfof law enforcement authoritiesor being subject to specific legal obligations as described above, the use of AI systems for making risk assessments in the context of private entities’ ordinary course of business and with the aim of protecting their own private interests, whilst the fact that those risk assessments may relate to the risk of criminal offences being committed merely as a purely accidental and secondary circumstance, is not deemed to be covered by the prohibition.