Point 1(b) of Annex III classifies as high-risk AI systems intended to be used for biometric categorisation according to sensitive or protected characteristics based on the inference of those attributes or characteristics. Such systems are deemed worthy of high-risk classification since they involve the processing of additional sensitive personal data and pose significant risks to other fundamental rights of natural persons , such as the right for private and family life , non-discrimination, and human dignity.
Article 3(40) AI Act defines a biometric categorisation system as ‘an AI system for the purpose of assigning natural persons to specific categories on the basis of their biometric data, unless it is ancillary to another commercial service and strictly necessary for objective technical reasons’. The biometric categorisation of an individual by an AI system is thus typically the process of establishing whether the biometric data of an individual belongs to a group with some pre-defined characteristic. It is not about identifying a specific individual o r verifying their identity, but about assigning an individual to a certain category. The notion of ‘biometric data’ is explainedSection 3.1.1. above.
Biometric categorisation may rely on categories of physical characteristics (e.g. facial features and form, skin colour) based on which persons are assigned to specific categories . Biometric categorisation may also be based on DNA or on behavioural aspects, such as keystroke analysis or a person’s gait23. Biometric categorisation excludes categorisation according to clothes or accessories, such as scarfs, hats, gloves, watches or crosses, as well as social media activity.
If a system is‘ancillary to another commercial service and strictly necessary for objective technical reasons’ it will fall outside the scope of the definition of biometric categorisation systems. Recital 16 AI Act explains that a purely ancillary feature is a feature that is intrinsically linked to another commercial service, meaning that the feature cannot, for objective technical reasons, be used without the principal service, and the integration of that feature or functionali ty is not a means to circumvent the applicability of the rules of the AI Act. i. According to sensitive or protected attributes or characteristics
According to point 1(b) of Annex III AI Act , only AI systems intended to be used for biometric categorisation according to sensitive attributes or characteristics will be classified as high-risk, in so far as those systems are not prohibited under the AI Act. Recital 54 AI Actclarifies that sensitive attributes or characteristics are those protected under Article 9(1) GDPR. They include attributes and characteristics on ‘racial or ethnic origin, political opinions, religious or philosophical beliefs, or trade union membership, genetic data, biometric data for the purpose of uniquely identifying a natural person, data concerning health or data concerning a natural person’s sex life or sexual orientation.’ The placing on the market, the putting into service for this specific purpose, or the use of biometric categorisation systems that categorise individually natural persons based on their biometric data to deduce or infer their race, political opinions, trade union membership, religious or philosophical beliefs, sex life or sexual orientation are prohibited by Article 5(1)(g) AI Act. Therefore, the attributes and characteristics of ethnic origin, genetic data, and data concerning health are here of key relevance. ii. Biometric categorisation must be based on the inference of those attributes or characteristics
To be classified as high-risk, the AI system must use biometric data to infer a sensitive or protected attribute or characteristic and then classify the individuals according to that attribute or that characteristic. This means that biometric data must be used as an input, while a sensitive or protected attribute or characteristic is the output of the AI system inferred from the biometric data.
a) Practical examples of AI systems falling within the high-risk use case of point 1(b)
- An AI system to categorise patients. The AI system is used to detect early symptoms of diseases that manifest themselves in mobility issues. The AI system captures patients’ gait, infers their health data based on captured gait data, and assigns those individuals to pre-defined categories (e.g., early stages, advanced stages of diseases).
- AI system to categorise passengers. The AI system is used in cameras installed at the airport to analyse passengers’ movement between terminals. The AI system captures travellers’ biometric data (e.g., gait, facial templates), infers biometric identifiers based on these captured data, and uses them to assign travellers into a category. Initial categorisation is based on inferred biometrics, e.g., ethnic origin. These categories are used, for example, to generate aggregated passenger profiles (e.g., solo passenger, passengers travelling as a family) or to support targeted operational measures.
b) Practical examples of AI systems falling outside the high-risk use case of point 1(b)
- AI system to categorise foreign persons crossing the EU border . The AI system is used in cameras installed at border crossing pointsand captures facial images to infer foreigners’ age and gender. The system is not high-risk because attributes such as age and gender are not considered as sensitive or protected under Article 9(1) GDPR. If age and gender are collected for the purposes of assessing the risk posed by foreigners entering into the EU territory, the AI system could be qualified as a high-risk system under point 7 (b) of Annex III, AI Act.
- AI system to categorise customers based on their gender to offer/improve personalised experience for customers. The AI system captures keystrokes, analyses them to assign a gender to the customer to offer/improve personalized experience(e.g., personalised advertising). The AI system deploys biometric categorisation (assigns a customer to a certain category based on their biometric data). However, the AI system infers gender, which is not a sensitive or protected attribute or characteristic under Article 9(1) GDPR.
- AI systems intended to be used for age estimation. The AI system prevents minors from, e.g.,
accessing online harmful content, (ii) accessing age-restricted services (such as online betting, personal loans, credit cards, investment platforms, mobile payment wallets), (iii) accessing physical premises not appropriate for children, or (iv) using v ending machines that provide products intended exclusively for persons over a legally defined age threshold (e.g., tobacco). The AI system captures keystrokes or/and facial features through a camera and assigns individuals to a specific age group on the basis of a comparison of the captured images/keystrokes patterns with characteristics of people in certain age groups and a probability prediction. The AI system infers age, which is not sensitive or protected attribute or characteristic under Article 9(1) GDPR.
- AI system intendedto be usedas a content moderation tool.The AI system scans text/pictures to identify illegal or inappropriate content. Although the system categorises content based on specific indications (e.g., extreme content that may reveal political opinions and be considered as a sensitive or protected attribute or characteristic), that categorisation is not made on the basis of biometric data.
- See e.g., the Article 29Working Party, Opinion 3/2012 on developments in biometric technologies , WP193, 27.4.2012, pp.16-17. The Group refers here to ‘soft recognition’ (p. 17), i.e. ‘detection of behaviour or specific needs of people’.