guidance

High-risk § 3.7.3

Point 7(b): AI systems intended to be used for risk assessment of persons seeking to enter or stay

Official source ↗

(393)

Point 7(b) of Annex III classifie s as high-risk AI systems intended to be used by or on behalf of competent public authorities or by Union institutions, bodies, offices or agencies to assess a risk, including a security risk, a risk of irregular migration, or a health risk, posed by a natu ral person who intends to enter or who has entered into the territory of a Member State.

(394)

The most important element in determining whether an AI system should be classified as high-risk pursuant to point 7(b) of Annex III is the intended purpose of the system to produce an individual risk score, category or flag that can influence decisions on entry, stay, detention, return or related measures. By contrast, AI systems that combine data streams into a higher-level summary (for example, by summing, averaging, or grouping data) to produce aggregate flow or staffing analytics, where outputs are not linked to identified individuals, should not be classified as high-risk pursuant to point 7(b) of Annex III.

(395)

AI systems that fall within the scope of the use case listed in point 7(b) of Annex III often amount to profiling because they assess a risk posed by a natural person. What matters is whether the system processes personal data in that context. In such a case, the system cannot benefit from the filtering mechanism in Article 6(3) AI Act. For example, AI systems that output a per-person risk flag or risk score, by using or inferring information about that person, conduct automated processing of personal data to evaluate or predict that person’s behaviour or movements and thus perform profiling. In contrast, if anonymised data are processed by the system so that no individual is identified or identifiable, the system should not be considered to perform profiling. Low-level aggregation or linkability with other datasets can still makeaggregated data (i.e. combined statistics on means, counts, rates) fall under the definition of personal data. AI systems that produce aggregate-only outputs that are never applied to identified or identifiable persons should be considered to fall outside the use case listed in point 7(b) of Annex III . However, if the system applies an aggregate indicator to a named or identifiable traveller to generate a risk flag or triage, its use involves personal data and constitutes profiling and therefore should be classified as high-risk pursuant to that provision.

(396)

An AI system falling with the use case listed in point 7(b) of Annex III may only benefit from the filter mechanism in Article 6(3) AI Act where it performs a narrow procedural or purely preparatory function that does not itself amount to profiling and does not materially influence the substance or outcome of an individual risk assessment. However, combined configurations are assessed as a whole, not individually (see Section 2.3. above).

a) Practical examples of AI systems falling within the use case of point 7(b)of Annex III

Person-level migration or security risk scoring

b) Practical examples of AI systems falling outside the use caseof point 7(b) of Annex III

Aggregate or cohort-level analytics