Point 6(e) of Annex III classifies as high-risk AI systems intended to be used by or on behalf of law enforcement authorities, or by Union institutions, bodies, offices or agencies in support of law enforcement authorities, for the profiling of natural persons as referred to in Article 3(4) LED in the course of the detection, investigation or prosecution of criminal offences. The inclusion of this use case in point 6 of Annex III affirms that profiling itself, when carried out through an AI system inter alia by law enforcement authorities in the criminal justice context, warrants a high-risk regulatory response, due to its potential to undermine the rights of due process, data protection, and equal treatment.
This provision applies to AI systems intended to be used by law enforcement authorities in identifying potential suspects or other persons of interest (e.g. victims) in various criminal contexts in the course of profiling of a natural person. Such systems typically analyse diverse types of data, such as geographic information, registry data, witness descriptions, or online activity, to generate profiles or flags that assist in crime detection, investigation or prosecution.
a) Practical examples of AI systems falling within the use case of point 6(e)
AI systems intended to be used in assisting the identification of potential suspects
- AI systems assisting in identifying potential suspects where a child is reported missing that analyse the neighbourhood, persons living in the neighbourhood that are on the sex offenders’ registry and their movements, as well as types of children they have abused earlier. The purpose of the AI system is to generate individualised risk inferences from behavioural patterns, thereby filtering persons as potential suspects. Such a system would involve profiling, since it performs the automated processing of personal data to evaluate and predict aspects of natural persons’ behaviour . Where the assessment of potential involvement in criminal activities is supported by past criminal behaviour and other verifiable facts , the system would not be solely based on profiling and assessing personal characteristics . It therefore falls within the use case of point 6(e) of Annex III AI Act , rather than the prohibition on predictive policing laid down in Article 5(1)(d) AI Act.
- AI systems assisting in identifying potential suspects based on descriptions of witnessesand other personal data. Such an AI system processes personal data to infer identity-linked traits and other personal aspects based on patterns, and to single out individuals as potential suspects. AI systems intended to be used for targeted online monitoring
- An AI system that monitorsonline posts to classify users based on sentiment analysis, posting patterns, and networks, and assigning flags for potential extremist affiliation. The system automatically analyses communication patterns, language use, and behavioural indicators to classify persons as potential radicalised subjects . The system performs profiling, since it evaluates and predicts natural persons’ characteristics and likely future behaviour for law enforcement purposes. This does not amount to social scoringas prohibited by Article 5(1)(c) AI Act , since the system is limited to targeted radicalisation monitoring within specific spheres or contexts.
b) Practical examples of AI systems falling outside the use case of point 6(e)
- AI systems analysing neighbourhoods and assigning criminality scores (predictive crime mapping systems) where they do not include profiling of individuals as per points 6(d) and
of Annex III.
- AI systems detecting suspicious transaction patterns that indicate money laundering or terrorist financing, even though that allows linking such transaction to personal profiles. However, borderline cases may arise where, in addition to focusing on transactions only (the structural, quantitative, or behavioural transactions data , e.g., transaction amounts, frequencies, intervals, geographic routing), such systems are combined with, or enriched by, personal data, thereby enabling the construction of individual profiles (e.g., names, account numbers).If the system assesses such personal characteristics, it should be classified as high-risk.
- An AI-enabled crime-linkage system analysing police crime reports and case files containing variables, such as time, place, modus operandi, suspect/victim descriptions, physical traits, clothing, vehicles, and socio-demographic indicators, with the purpose to identify patterns and similarities across different incidents, linking them into possible ‘series’ committed by the same offender(s). Its main outputs are alerts, crime link hypotheses, and forecasts of likely future offences (not linked to a particular individual).