Point 5(a) of Annex III classifies as high-risk AI systems intended to be used by public authorities or on behalf of public authorities to evaluate the eligibility of natural persons for essential public assistance benefits and services, including healthcare services, as well as to grant, reduce, revoke, or reclaim such benefits and services. Such use may result in the system taking or support ing decisions by or on behalf of public authorities that evaluate the eligibility of natural persons for essential public assistance benefits and services, including healthcare services, as well as grant, reduce, revoke, or reclaim such benefits and services. If not developed or used correctly, AI systems intended to be used for such purposes may include biases or errors, possibly falsely evaluating a natural person as ineligible to receive benefits, singling them out for further inspections, or putting them in a less prioritised category. To address such risks, such systems are classified as high-risk and subjected to safeguards to ensure that they work as intended and are accurate, safe, and do not discriminate.
For such an AI system to be classified as high-risk pursuant to point 5(a) of Annex III AI Act it must be intended to be used by or on behalf of public authorities : (i) to evaluate the eligibility of natural persons for essential public assistance benefits and services, including healthcare services; or (ii) to grant, reduce, revoke, or reclaim such benefits and services; or (iii) for both purposes.
The conditions that an AI system must fulfil to be classified as high-risk pursuant to point 5(a) of Annex III and related concepts are analysed in more detail in the following subsections. i. The concept of ‘public authorities’ and ‘on behalf of authorities’
Point 5(a) of Annex III provides that the AI system must be intended to be used by public authorities or on their behalf.
The AI Act does not define the concept of public authorities. Inspiration may be drawn from the CJEU case-law on the concept of a public authority . In its case law, the CJEU has defined the concept of public authorit ies (for the purposes of Directive 2003/4) as administrative authorities that form part of the public administrationor the executive of the State at any level, which includes legal persons governed by public law set up by the State and which it alone can decide to dissolve60. Additionally, entities that are governed either by public law or private law are also administrative authorities in case they perform services of public interest and are ‘vested with special powers beyond those which result from the normal rules applicable in relations between persons governed by private law’61. EU institutions, bodies, and agencies are also public authorities.
As regards the terms ‘on behalf of” used in point 5(a) of Annex IIIreference is made toSection 2.5. above. ii. The concept of ‘essential public assistance benefits and services’
Recital 58 AI Act clarifies that essential public assistance benefits and services are those benefits and services for which natural persons apply, or which natural persons receive that are necessary for them to fully participate in society or to improve their standard of living. Such natural persons will typically be dependent on the benefits and services for which they applyand be in a vulnerable position in relation to the competent public authorities. The recital gives an illustrative list of the main types of such public services and benefits that could be considered to fall within the scope of point 5(a) of Annex III. These include62: 1. Healthcare services (such as long-term care services); 2. Social security benefits (such as sickness benefits; maternity and equivalent paternity benefits; invalidity benefits; old-age benefits; survivors’ benefits; benefits in respect of accidents at work and occupational diseases ; death grants; unemployment benefits; pre-retirement benefits; family benefits; long-term care benefits; minimum income benefits; housing benefits; benefits to access essential services (e.g. water, energy, digital communications)); 3. Social services (such as social assistance services, employment and training services, housing support and social housing, childcare).
Certain public support measures, which are not public benefits or services as such do not qualify as essential according to the use case listed in point 5(a) of Annex III. For example, tax remission is excluded from that use case , since it is not a service or benefit as such, even if it may provide financial support for beneficiaries. iii. The concept of evaluating the eligibility of a natural person
There are various types of public assistance benefits and services provided in the Member States, all with differing ways to evaluate a natural person’s eligibility to claim them. The evaluation of eligibility should be understood as evaluating and deciding on the possibility of a natural person to receive the benefit or service, but also in terms of the level of benefit or service received. Such an evaluation is generally based on some pre-defined criteria and will be carried out prior to the decision to grant or deny the benefits of services at issue. Once eligibility is determined, the actual decision to grant or deny the benefits or services will be taken . The evaluation for eligibility may also be carried out after a preliminary decision has been taken to grant or deny such benefits and services. In such a situation, the decision to grant or deny will be confirmed by evaluating the person’s eligibility. This may in turn lead to situations in which a natural person has received too many assistance benefits and needs to pay them back or has received too few assistance benefits and they need to be supplemented. iv. The concept of ‘grant or denial’ of public assistance benefits and services
A range of decisions concerning the grant or denial of public assistance benefits and services may fall within the use case listed in point 5(a) of Annex III . This follows from the use of the terms ‘grant, reduce, revoke, or reclaim’, which includes all possible scenarios related to such decisions. For the sake of clarity, it should be noted that Recital 58 AI Act also mentions the denial of public assistance benefits and assistance. While this is not explicitly stated in point 5(a) of Annex III, a holistic approach suggests that an AI system intended to be used to deny a natural person public assistance benefits and/or services (and not just a reduction, revocation or reclamation of previously granted benefits and/or services) also falls within the use case listed therein. v. The link between the eligibility evaluation and the grant or denial of essential public assistance benefits and services
AI systems intended to be used by or on behalf of a public authority solely to evaluate the eligibility of a natural person to receive essential public assistance benefits and services will be classified as high-risk, even if the system is not subsequently used to grant or deny those benefits and services. It is in fact often the case that AI systems are used solely for this purpose, without the system taking decisions regarding the grant or denial of benefits or services. Similarly, an AI system intended to be used solely to grant, deny, reduce, revoke, or reclaim such benefits and services also falls within the use case, without the need for that system to be involved in the prior eligibility evaluation.
AI systems intended to be used by or on behalf of public authorities to grant a natural person essential public assistance benefits and services following a positive evaluation of their eligibility, thereby improving their standard of living and livelihood, will also be classified as high-risk pursuant to point 5(a) of Annex III . That remains true if the grant follows an incorrect evaluation of a natural person’s eligibility for such benefits and services, which consequently may lead to the denial, reduction, revocation, or reclamation of the benefits and services previously granted, having a negative impact on the person’s standard of living and livelihood.
a) Practical examples of AI systems falling within the high-risk use case of point 5(a)
Automated decisions determining eligibility
- An AI system intended to decide on the grant of unemployment benefits in individual cases. The system analyses whether the person qualifies for the benefit, during which period and in which amount, and makes a recommendation to a case handler about the eligibility and the receiving of the benefits. Such an AI system is high-risk, because it is intended to evaluate a natural person’s eligibility, and makes a recommendation on the granting or denial of these benefits.
- An AI system intended to assess a natural person’s eligibility for social benefits following an application. The system processes application data and cross-references external databases, after which it generates a decision on the applicant’s eligibility for social benefits based on the given data. Such an AI system is high-risk, because it is intended to evaluate a natural person’s eligibility and generates a decision on their elgibility.
- An AI system intended to be used by public administrations to evaluate a natural person’s eligibility for social housing or unemployment benefits through a scoring system that is not a prohibited practice covered by Article 5 (1)(c) AI Act . The system draws on demographic, behavioural, and financial data to predict the likelihood of fraud or long-term dependency. Such an AI system is high-risk, since it aids in the evaluation of a natural person’s eligibility and materially influences decision about whether the applicant will receive the benefits and services. Prioritisation
- An AI system intended to evaluate applications for social housing. The system evaluates the profiles of applicants for eligibility, urgency level and prioritisation for specific social housing units based on income, household size, age, employment status a nd location. The system can substitute or influence human decision-making and is high-risk, since it has a direct impact on access to essential housing benefits.
- An AI system intended to prioritise the allocation of home-care services. Such systems may be used by local authorities to determine the distribution and prioritization of limited in-home care hours among eligible persons. Such an AI system is high-risk, since it materially influences the access to essential care services for elderly persons or persons dependent on such care, thereby potentially affecting their fundamental rights and well-being.
- An AI system intended to assess the eligibility of applications for access to unemployment support or income support schemes , administered by public authorities, and prioritise th ose applications accordingly in a context of limited resources or processing capacity . Such an AI system is high-risk, since it evaluates applications for essential public assistance benefits and services and it materially influences the granting, timing or effective access to such benefits and services through prioritisation. Supportive tools for human review determining eligibility or access
- An AI system intended to analyse applications for essential public assistance benefits submitted by natural persons (e.g. sickness benefits) in order to detect potential irregularities and/or inaccuracies, or fraud indicators and to flag applications for further investigation by a public authority. When an application is flagged, a human case handler takes over to assess whether the claim is fraudulent. Such an AI system is high-risk, since it is used in the context of evaluating eligibility for essential public assistance benefits and materially influences access to such benefits. In particular, flagging determines whether applications are subject to further scrutiny, delay, suspension or potential refusal and is therefore instrumental to decision s affecting the granting, continuation or effective access to those benefits.
- A chatbot intended to answer legal questions of a case handler that are specifically related to the evaluation of an application of a natural person to receive essential care benefits or services administered by a public authority. The case handler can grant or refuse the benefits to which the natural person applied, while remaining responsible for the final decision, based on the answers given by the chatbot that are personalised to the specific case/individual eligibility assessment. Such an AI system is high-risk, since the case handler’s decision on the eligibility of the natural person to receive care benefits or services is materially influenced by the legal answers given by the chatbot.
- An AI system is intended to assess applications for healthcare in order to determine whether an in-depth examination of the application should be carried out by an external body . Even if the decision to forward is taken by a case handler , such an AI system is high-risk, since the system preliminary assessment acts as a decisive step for further examination determining the access to healthcare services. Proactive invitation with an eligibility check
- An AI system intended to be used by public authorities for health preventive screenings to suggest personalised invitations for preventive screening (such as mammograms), based on population segmentation, performing also an evaluation of the eligibility for the preventive screening. Such an AI system is high-risk, since it evaluates the eligibility of a person to do a preventive screening (a healthcare service), even if this is done through a proactive invitation.
b) Practical examples of AI systems falling outside the high-risk use case of point 5(a)
Proactive invitations without an evaluation of the eligibility
- AI systems intended to proactively identify persons in need of preventive care. Such an AI system falls outside the use case of point 5(a) of Annex III, since it only identifies persons in need of preventive care and does not determine the eligibility of the identified persons, provided that the system’s outputs are not used to prioritise, exclude or otherwise influence access to such a service.
- AI systems intended to match persons with the public support services that fit them best based on their needs, using anonymised data to suggest and pre-fill applications for support services. Such an AI system falls outside the use case of point 5(a) of Annex III, since it is not intended to evaluate the eligibility of a natural person, but to support the delivery of public support services. Case-handler allocation
- AI systems intended to merely inform by which body or competent person an application for essential public assistance benefits is (further) to be assessed. Such an AI system falls outside the use case of point 5(a) of Annex III, since it only recommends who should assess the application, without an actual evaluation of the eligibility. Eligibility checks and reimbursement of costs of legal persons
- AI systems intended to assess applications or claims from companies for reimbursement of costs, but not from natural persons. Such an AI system falls outside the use case of point 5(a) of Annex III, since it does not assess applications or claims from natural persons.
c) Practical examples of AI systems falling within the use case but exempted under the filter mechanism of Article 6(3)
Chatbots answering factual questions of a case handler
- A chatbot intended to answer factual questions of a case handler that are related to the evaluation of an application of a natural person to receive healthcare benefits (such as the age of the natural person). The case handler can grant or deny the benefit s to which the natural person applied, based on the answers given by the chatbot. Such an AI system falls within the use case of point 5(a) of Annex III , since the case handler’s decision on the eligibility of the natural person to receive healthcare benefits depends on the answers given by the chatbot. However, since the system only provides existing and factual information in a structured manner, without evaluating this information for the eligibility of a natural person’s application for healthcare benefitsand , the case handler must review the information made available by the chatbot to ensure that the decision is not based solely on the output of the AI system, it falls under the exception for AI systems intended to perform narrow procedural tasks listed in Article 6(3)(a) and (d) AI Act. Supportive tools for human review
- AI systems intended to summarise medical reports which will be used by a case handler as the basis for their decision to grant or deny healthcare benefits. Such an AI system fall s within the use case of point 5(a) of Annex III, since it is intended to be used for the access to healthcare services. However, since the system is limited to summarising the information in the reports and the case handler must review the relevant information in the reports before taking a decision, it falls under the exception for AI systems intended to perform narrow procedural tasks listed in Article 6(3)(a) and 6(3)(d) AI Act. Language tools
- AI systems intended to translate applications for public assistance services made in a foreign language. Such an AI system fall s within the use case of point 5(a) of Annex III , since it is intended to be used for the evaluation of a natural person’s application for public assistance benefits. However, since the translation of applications for assistance benefits and services is an unavoidable step in the process, but it is not decisive for the evaluation of the eligibilityof those applications, the system falls under the exceptions for AI systems intended to perform narrow procedural and preparatory tasks listed in Article 6(3)(a) and (d) AI Act.
- AI systems intended to be used in conversations between a case handler and a natural person who wants to apply for public assistance services and benefits by converting speech into text . The system creates a summary of the conversation that will be used to make a decision concerning the eligibility of a natural person. Such an AI system fall s within the use case of point 5(a) of Annex III , since it is linked to the evaluation of a natural person’s application for public assistance benefits and services. However, in so far as the case handler is present during the conversation and is able to check the summary made by the system , so that the output is only a supporting but not decisive element for his or her decision, the system falls under the exception for AI systems intended to perform only narrow procedural tasks listed in Article 6(3)(a) and 6(3) (d) AI Act.
- Case C-279/12,Fish LegalEU:C:2013:853, para 51: ‘Entities which, organically, are administrative authorities, namely those which form part of the public administration or the executive of the State at whatever level, are public authorities for the purposes of Article 2(2)(a) of Directive 2003/4. This first category includes all legal persons governed by public law which have been set up by the State and which it alone can decide to dissolve. ’
- Case C-279/12,Fish Legal EU:C:2013:853, para 52: ‘The second category of public authorities, defined in Article 2(2)(b) of Directive2003/4, concerns administrative authorities defined in functional terms, namely entities, be they legal persons governed by public law or by private law, which are entrusted, under the legal regime which is applicable to them, with the performance of servi ces of public interest, inter alia in the environmental field, and which are, for this purpose, vested with special powers beyond those which result from the normal rules applicable in relations between persons governed by private law .’
- Services and benefits are mentioned in Regulation (EC) No 883/2004 on the coordination of social security systems, 2019 Council Recommendation on access to social protectionfor workers and the self-employed, the 2023 Council Recommendation on adequate minimum income for active inclusion , and the 2008 Commission Recommendation on the active inclusion of people excluded from the labour market.