Prohibitions § 7.2.2

The prohibition in Article 5(1)(f) AI Act is limited to emotion recognition systems in the ‘areas of workplace and educational institutions’. As clarified in recital 44 AI Act, this limitation is meant to address the imbalance of power in the context of work or education.

The notion of ’w orkplace’ should be interpreted broadly . That notion relates to any specific physical or virtual space where natural persons engage in tasks and responsibilities assigned by their employer or by the organisation they are affiliated to, for example in case of self-employment. This includes any setting where the work is performed and can vary widely based on the nature of the job, spanning from indoor office spaces, factories and warehouses to publicly accessible spaces like shops, stadiums or museums, to open-air sites or cars , as well as temporary or mobile work sites. This is independent from the status as an employee, contractor, trainee, volunteer, etc.¹⁶¹ The notion of ‘workplace’ in Article 5(1)(f) AI Act should also be understood to apply to candidates during the selection and hiring process , consistently with other provisions of the AI Act addressing the placing on the market, putting into service or use of AI systems in the area of employment, workers management and access to self-employment, since there is an imbalance of powers and the intrusive nature of emotion recognition may already apply at the recruitment stage. For example,

• Using webcams and voice recognition systems by a call centre to track their employee’s emotions, such as anger, is prohibited.If only deployed for ¹⁶² personal training purposes, emotion recognition systems are allowed if the results are not shared with HR responsible persons and cannot impact the assessment, promotion etc. of the person trained, provided that the prohibition is not circumvented and the use of the emotion recognition system does not have any impact on the work relationship.
• Using voice recognition systems by a call centre to track their customers emotions, such as anger or impatience, is not prohibited by Article 5(1)(f) AI Act (for example to help the employees cope with certain angry customers).
• AI systems monitoring the emotional tone in hybrid work teams by identifying and inferring emotions from voice and imagery of hybrid video calls , which would typically serve the purpose of fostering social awareness, emotional dynamics management, and conflict prevention, are prohibited.
• Using emotion recognition AI systems during the recruitment process is prohibited.
• Using emotion recognition AI systems during the probationary period is prohibited.
• Using cameras by a supermarket to track its employees’ emotions, such as happiness, is prohibited.
• Using cameras by a supermarket or a bank to detect suspicious customers, for example to conclude that somebody is about to commit a robbery , is not prohibited under Article 5(1)(f) AI Act, when it is ensured that no employees are being tracked and there are sufficient safeguards. b) ‘Education institutions’

The reference to education institutions is broad and should be understood to include both public and private institutions. There is no limitation as regards the types or ages of pupils or students or of a specific environment (online, in person, in a blended mode,¹⁶³ etc.). For example, education and training institutions at all levels fall under the scope of the prohibition in Article 5(1)(f) AI Act, including vocational schools, i.e. schools where students learn skills involving the use of their hands¹⁶⁴ and continuous training¹⁶⁵. Education institutions are normally accredited or sanctioned by the relevant national education authorities or equivalent authorities. A key feature is that education institutions may provide a certificate (respectively participation is a precondition for obtaining a certificate). The prohibition should be understood to also apply to candidates during the admissibility process. For example,

• An AI-based application using emotion recognition for learning a language online outside an education institution is not prohibited under Article 5(1)(f) AI Act. By contrast, if students are required to use the application by an education institution, the use of such emotion recognition system is prohibited.
• An education institution using AI-based eye tracking software when examining students online to track the fixation point and movement of the eyes (gaze point, e.g., to detect if unauthorized material is used) is not prohibited, because the system does not identify or infer emotions . B y contrast, if the system is also used to detect emotions, such as emotional arousal and anxiousness, this would fall within the scope of the prohibition.
• Using an emotion recognition AI system by a n education institution to infer the interest and attention of students is prohibited. By contrast , i f only deployed for learning purposes in the context of a role-play (for example, for training actors or teachers), emotion recognition systems are allowed if the results cannot impact the evaluation or certification of the person being trained.
• Using an emotion recognition AI system by an education institut ion during admissibility tests for new students is prohibited.
• Using an AI system that allows to capture students talking to each other via their phones or other channels during online lectures by an education institution is not prohibited, since it does not infer emotions. By contrast, if the system is also used to detect emotions, such as emotional arousal, anxiousness and interest, this would fall within the scope of the prohibition.
• An education institution employing an emotion recognition AI system on both teachers (workplace) and students (education) is prohibited.