Point 6(d) classifies as high-risk AI systems intended to be used by or on behalf oflaw enforcement authorities, or by Union institutions, bodies, offices or agencies in support of law enforcement authorities, to assess the risk of (a) concrete person(s) offending or reoffending , provided that assessment is not based solely on profiling or evaluating their personality traits, or by looking at their past criminal behaviour or the past criminal behaviour of the groups to which that person belongs. In other words, the use case in point 6(d) includes AI systems intended to be used to assess the risk of a person committing a crime or reoffending, without prejudice to the prohibition set out in Article 5(1)(d) AI Act. Each of the conditions of the use case listed in point 6(d) of the Annex III AI Act shall be examined in the subsections below.
ii. The concept of ‘not solely on the basis of” (i) profiling, (ii) assessment of personality traits and characteristics, and (iii) assessment of the past criminal behaviour of natural persons or groups as alternatives
The use case listed in point 6(d) of Annex III captures predictive and behavioural AI systems that are intended to be used to assess the risk of a natural person offending or re-offending. It explicitly excludes AI systems that are based solely (i) on profiling as per Article 3(4) LED , (ii) on the assessment of personality traits and characteristics , which are already prohibited under Article 5(1)(d) AI Act (see Section 5 of the Guidelines on prohibited artificial intelligence practices)77or
on the assessment of past criminal behaviour of natural persons or groups . Recital 59 AI Act clarifies that the use cases listed in point 6(d) of Annex III are to be interpreted as alternative. If at least one of these use cases is present, the AI system qualifies as high-risk, unless it falls under the prohibition set out in Article 5(1)(d) AI Act.
The use of an AI system to assess the risk of a natural person offending or re-offending is not based solely on profiling where that system is used to support the human assessment of the involvement of a person in a criminal activity, which is already based on objective and verifiable facts directly linked to a criminal activity. As recital 42 AI Act clarifies, this will be the case where a reasonable suspicion already exists in respect of the natural person concerned. Paragraphs 199 to 206 of the Guidelines on prohibited artificial intelligence practices contain further guidance on the notion of ‘solely” in relation to profilingunder Article 5(1)(d).
a) Practical examples of AI systems falling within the use case of point 6(d)
AI systems used when processing detained persons to assess the risk of offending/reoffending
- An AI system intended to be used during the initial processing of detained youth suspects by police to calculate a risk score that indicates the likelihood of reoffending, which is then used to make a decision on measures to be taken. The AI system does not autonomously make decisions; the outcome is considered alongside other information, and final decisions are made by the competent authorities. The AI system is intended to be used in support of law enforcement authorities for assessing the risk of a natural person re-offending and this assessment is done not solely on the basis of the profiling of natural persons as referred to in Article 3(4) LED, or to assess personality traits and characteristics or past criminal behaviour of natural persons or groups, as there are verifiable facts directly linked to a criminal activity (the assessment is performed post-detention). AI systems used by probation officers
- An AI system intended to be used by probation officers to support assessments informing decisions on parole or conditional release, by evaluating factors such as criminal record, behaviour and progress during the custodial sentence, and other case-relevant supervision-related information . Like the example above, there are verifiable facts directly linking a person to a criminal activity(the person having already been convicted of a criminal offence). AI systems used by penitentiary institutions
- An AI system used by a penitentiary institution to predict the likelihood of an offender re-offending based on prior criminal record, violence within the prison facilities, or socio-economic data. Like the examples above, there are verifiable facts directly linking a person to criminal activity (the person having either already been convicted of a criminal offence or being detained on suspicion of having committed such an offence). AI systems used by criminal courts to assess the risk of offending/reoffending
- An AI-based risk assessment system intended to be used in criminal courts to estimate the likelihood of recidivism, violent reoffending, or the commission of further serious offences. Since such a risk assessment system involves profiling, the exceptions of Article 6(3) AI Act do not apply.
b) Practical examples of AI systems falling outside the use case of point 6(d)
Location focused AI-systems
- AI systems predicting locations and timeframes of likely crimes (e.g., burglary risk in a neighbourhood), without linking the prediction to identifiable individuals. The use case in point 6(d) concerns risk assessments of natural persons, not geospatial predictions.
c) Practical examples of AI systems falling within the use case but exempted by the filter mechanism in Article 6(3) AI Act
- An AI system that checks whether all required fields are completed in a probation officer’s risk assessment form or flags inconsistencies in entered data (e.g., mismatched dates of offence or custody period). Such an AI system performs a preparatory task f or the risk assessment and does not evaluate personal characteristics. It is also intended for a narrow procedural task (Article 6(3)(a) AI Act), since it only flags inconsistencies and would in any event fall under the exception for AI systems intended t o perform a preparatory task listed in Article 6(3)(d) AI Act.
- An AI system that collects and pre-sorts prior convictions, probation reports, and other records to prepare a case file for human officers. Such a system performs a preparatory task for risk assessment and therefore qualifies for an exemption under Article 6(3)(d) AI Act.
- Article 5(1)(d) AI Act prohibits AI systems assessing or predicting the risk of a natural person committing a criminal offence based solely on profiling or assessing personality traits and characteristics.