(214)
Article 5(1)(d) AI Act provides that the prohibition does not apply to AI systems used to support the human assessment of the involvement of a natural person in a criminal activity, which is already based on objective and verifiable facts directly linked to a criminal activity. In such a case, the individual crime risk assessments and predictions would also not be based solely on profiling or the assessment of personal characteristics and would therefore not be prohibited. Examples of AI systems falling outside the scope of the prohibition for th is reason include:
- The use of an AI system for profiling and categorisation of actual behaviour, such as reasonably suspicious dangerous behaviour in a crowd that someone is preparing and is likely to commit a crime, and there is a meaningful human assessment of th e AI classification. In this case, the risk assessment made by the human with the support of AI is not solely based on the personal traits or the profiling , but on objective and verifiable facts linked to the threatening criminal behaviour of that person that has been reviewed by a human before action taken.
- The police is investigating the risk of a possible armed robbery and suspects two individuals. Several verifiable and objective facts are present on which that suspicion is based, such as verifiable participation and dialogues in dark web chat groups for purchasing arms. An AI system com bining geospatial predictive or place-based policing information and Automated Number Plate Registration (ANPR) information of vehicles belonging to the suspects supports the human assessment in the investigation based on verifiable and objective facts directly linked to a specific criminal activity.
- The use of an AI system that assesses the risk whether a prisoner should receive the benefit of an early release. The AI profile of the affected person or assessment of their personality traits and characteristics only support the human assessment of objective and verifiable facts related to past criminal offences and demonstrated behaviour relevant to rehabilitation.
- A judge conducts a pre-trial detention hearing for a person accused of a serious criminal offence to assess whether non-custodial measures can be applied. The decision is based on an assessment of the existence of valid grounds to imposed pre-trial detention, such as the likelihood that the suspect or accused person will commit another offence if not detained or that they will abscond or obstruct the proper conduct of the investigation. To assist in this process, the judge uses an AI risk assessment tool trained on data including past criminal history of individuals in similar cases, as well as factors such as age group, social behaviour, income, and employment status.
- AI system is used to support the assessment of a human officer toassess the risk of an individual serving a non-custodial sentence violating release conditions or absconding based on past criminal behaviours and objective facts that give grounds for suspicion such as adherence to conditions of release, psychological ass essment outcomes and recommendations from other community services the individual may be using. Based on this information, the officer decides whether to maintain the status quo or revise the conditions of release.
- AI systems used by custom authorities to assess the risk of goods entering the EU not complying with the legislation applicable at the border (e.g., which may include bans on import of illicit drugs, export sanctions contravention or other illegal activity) to identify situations w here a customs control should be carried out. The AI system assesses objective and verifiable information provided to the customs related to the goods and their supply chains (e.g., nature and value of the goods, container number, means of transport for co ncealment of other goods, prior knowledge relating to the compliance of goods of the particular description and origin with requirements for their importation to or exportation from the Union). In certain cases, it may also process information about the pr ior involvement of the importer or exporter in irregularities related to import of goods, their affiliation to criminal organizations or a criminal record for drug trafficking. Such systems are out of scope of the prohibition because any prediction of a likelihood of a natural person to be involved in an import or export of illicit goods is not solely based on profiling, but on objective and verifiable information related to the goods and the importer or exporter’s prior involvement in criminal activity and subject to a human review to determine whether or not the situation requires a customs control or risk mitigation action.