Risk assessments to assess or predict the risk of an individual committing a criminal offence are often referred to as individual ‘crime prediction’ or ‘crime forecasting’. While there is no general ly agreed definition of ‘crime prediction’or ‘crime forecasting’,129 these terms refer in general to a v ariety of advanced AI technologies and analytical methods applied to large amount ofoften historical data (including socio-economic data, but also police records, etc .) which, in combination with criminology theories, are used to forecast crime as a basis to inform police and law enforcement strategies and action to combat, control, and prevent crime.130
Crime prediction AI systems identify patterns within historical data, associating indicators with the likelihood of a crime occurring, and then generate risk scores as predictive outputs. For example, such systems may be used for planning police task forces, for monitoring high-risk situatio ns, and for conducting controls of persons predicted as likely (re-)offenders. Such systems bring opportunities for law enforcement authorities, especially those with scarce resources, increasing efficiency, and enabling a proactive approach for detecting, deterring , and anticipating criminal offences.131 However, such use of historical data on crimes committed to predict other persons’ future behaviour may perpetuate or even reinforce bias es, and may result in crucial individual circumstances being ‘overlooked’ when these circumstances are not part of the data set or considered in the algorithms on which the particular AI system operates. This may also undermine public trust in law enforcement and the justice system in general.132
Such risk assessments and predictions are, in principle, forward-looking and concern future criminal offences (not yet committed ) or crimes that are assessed as a risk of being committed at the moment , including in cases of an attempt or preparatory activities undertaken to commit a criminal offence.133 They can be made at any stage of the law enforcement activities, such as during prevention and detection of crimes, but also during the investigation, prosecution and execution of criminal penalties (including when judicial authorities assess the risk of re-offending e.g., in the context of making decisions on the imposition of pre-trial detention) as well as part of the individuals’ plan for re-integration into society after serving a criminal sentence.134
The prohibition in Article 5(1)(d) AI Act does not outlaw crime prediction and risk assessment practices as such. It only applies to AI systems for making risk assessments to assess or predict the risk of a natural person committing a criminal offence, where also the third condition referred to above is met . Moreover, as noted, the prohibition does not apply in the situations described in the express exclusion contained in the last phrase of Article 5(1)(d) AI Act.