To fall within the use case listed in point 6(a) of Annex III, the AI system must fulfil the following cumulative conditions:
The system must be intended to be used by or on behalf of law enforcement authorities, or by Union institutions, bodies, offices or agencies in support of law enforcement authorities or on their behalf;
The system must be intended to be used to assess the risk related to a natural person as a victim (i.e. not an object, geographic location, event, or organisation); and
The person must be a potential future victim of criminal offences (notother risks such as accidents or administrative breaches).
AI systems intended to be used to assess the risk of a natural person becoming the victim of criminal offences are typically designed to support preventive and protective law enforcement strategies. Such AI systems analyse various personal and contextual data related to individuals in order to assess their risk or vulnerability to certain criminal situations. They process information , such as personal histories, official reports, social or family circumstances , to produce evaluations of the likelihood that an individual may become a victim of a specific crime.
The AI Act does not define the term ‘victim’. For the purposes of point 6(a)76 Annex III AI Act , this term can be understood to refer to a natural person who is inferred or identified by an AI system as being exposed to a probability or risk of sufferingharm, including physical, mental or emotional harm or economic loss, resulting from conduct that, if materialised, constitutes a criminal offence.
The classification of such AI systems as high-risk is based on the potential impact those systems can have on safety, security or fundamental rights. Such AI systemscan be both useful in protecting the fundamental rights of potential victims (e.g. to protect a person from bodily harm ) while improper functioning or use could trigger increased surveillance, welfare checks, damage to reputation, cause negative psychological consequences or stigmatisation.The proper functioning of such systems is therefore of material importance, since flaws or misuse can lead to a risk of underassessment and false negatives that, combined with overreliance by law enforcement authorities, may result in withholding of preventive measures or protection (e.g. in case of assessing the protection needed by an alleged victim of gender-based violence).
a) Practical examples of AI systems falling within the use case of point 6(a)
Domestic violence risk assessment systems
- Domestic violence risk assessment systems intended to be used to analyse victim statements, police reports, previous incidents, restraining order data, social services records and other data to determine the level of risk faced by a (potential) victim of domestic violence should be considered to fall in the use case listed in point 6(a) of Annex III . Such AI systems are a classic example of a tool that is used by law enforcement authorities to assess the risk of a natural person becoming the victim of a criminal offence. They influence critical interventions, including police protection measures and legal actions, which can either prevent harm from arising or, if flawed, leave victims vulnerable to further abuse. The use of such AI systems involves profiling, so that they cannot benefit from thefiltering mechanism laid down in Article 6(3) AI Act.
- An AI-based risk assessment system intended to be used by judicial authorities in criminal court proceedings to estimate the probability of severe or repeated domestic intimate partner violence, for instance when issuing or maintaining restraining orders. As such risk assessment involves profiling, the exemption under Article 6(3) AI Act cannot be applied. Vulnerability exploitation risk assessment systems
- Human trafficking vulnerability detection systems intended to be used to assess a risk of an individual being trafficked, especially in cross-border, labour exploitation or migration contexts, or in the case of young individuals from particularly vulnerable backgrounds , should be considered to fall in the use case listed in point 6(a) of Annex III . Such systems aggregate indicators such as age , lack of valid documentation, traveling with an unrelated adult, signs of control by others, and links to high-risk sectors, to produce a score indicating the likelihood of future human trafficking. Those systems involve individual risk assessment and have potential consequences for fundamental rights. The use of such AI systems involves profiling, so that they cannot benefit from the filtering mechanism laid down in Article 6(3) AI Act.
b) Practical examples of AI systems falling outside the use case of point 6(a)
AI systems predicting location-focused risks
- AI systems intended to be used to analyse the risk that a specific environment/location will see offending and victimisation (i.e., people in certain areas will become victims of crime) should be considered to fall outside the use case listed in point 6(a) of Annex III . Being environment/location-focused, such a system do es not explicitly assess risks to specific persons (victims) except by virtue of them being in a risky area. However, if the system is intended to be used to predict the potential risk of a specific person or persons committing a crime in that area, such a system would fall within the use case listed in point 6(d) of Annex III.
- Situational awareness and risk assessment AI system s intended to be used to support law enforcement during the initial stages of responding to a reported crime scene should be considered to fall outside the use case listed in point 6(a) of Annex III. Such a system can be deployed via a robot or drone, and is designed to process visual and contextual inputs to detect critical elements, such as presence of armed individuals, weapons or environmental hazards. Since the aim of the system is to evaluate the immediate environment, rather than assessing the risks related to specific individuals based on personal or contextual data, it should not be classified as high-risk on that basis alone. However, if the system also had biometric capabilities, it could fall within the use case listed in point 1 of Annex III (see Section 3.1. above). AI systems predicting risks of accidents or administrative breaches
- AI systems intended to be used to identify road segments or times where accidents are likely to occur, thus where people are at risk of becoming victims of road traffic accidents, should be considered to fall outside the use case listed in point 6(a) of Annex III. For such a system to fall within that use case, it should be intended to be used to analyse the risk of becoming the victim of criminal offences and not of accidents . Moreover, such systems are location-focused and do not analyse risks linked to concrete persons.
- AI-based video analytics intended to be used to predict overcrowding, stampedes, or structural hazards at large gatherings should be considered to fall outside the use case listed in point 6(a) of Annex III.
- AI flood-risk and wildfire-spread models intended to be used to identify populations at risk of becoming victims of accidents/catastrophes should be considered to fall outside the use case listed in point 6(a) of Annex III . See also Section 3.6.5. on the use case listed in point 5(d) of Annex III.
- Article 2(1)(a) of Directive 2012/29/EU (the Victims' RightsDirective; Directive 2012/29/EU of the European Parliament and of the Council of 25 October 2012 establishing minimum standards on the rights, support and protection of victims of crime, and replacing Council Framework Decision 2001/220/JHA. OJ L 315, 14.11.2012, pp. 57–73)provides a definition of the term ‘victim’. It should,however,be noted that the definition in the Victims' Rights Directive is retrospective, in that it refers to a person who has already suffered harm,whereasthe use case in point 6(a) of Annex III to the AI Act is forward-looking.Similarly, or the definition of ‘victim” for the purposes of gender based violence can be found in Directive (EU) 2024/1385 on combating violence against women and domestic violence. OJ L, 2024/1385, 24.5.2024.