The following AI practices shall be prohibited:
the placing on the market, the putting into service or the use of AI systems for the evaluation or classification of natural persons or groups of persons over a certain period of time based on their social behaviour or known, inferred or predicted personal or personality characteristics, with the social score leading to either or both of the following:
detrimental or unfavourable treatment of certain natural persons or groups of persons in social contexts that are unrelated to the contexts in which the data was originally generated or collected;
detrimental or unfavourable treatment of certain natural persons or groups of persons that is unjustified or disproportionate to their social behaviour or its gravity;
Several cumulative conditions must be fulfilled for the prohibition in Article 5(1)(c) AI Act to apply:
The practice must constitute the ‘placing on the market’, the ‘putting into service’ or the ‘use’ of an AI system.
The AI system must be intended or used for the evaluation or classification of natural persons or groups of persons over a certain period of time based on: (a) their social behaviour; or (b) known, inferred or predicted personal or personality characteristics.
The social score created with the assistance of the AI systemmust lead or be capable of leading to the detrimental or unfavourable treatment of persons or groups in one or more of the following scenarios: (a) in social contexts unrelated to those in which the data was originally generated or collected; and/or (b) treatment that is unjustified or disproportionate to their social behaviour or its gravity.
For the prohibition in Article 5(1)(c) AI Act to apply , all three conditions must be simultaneously fulfilled. The first condition, i.e. the placing on the market, the putting into service or the use of the AI system, has been already analysed in section 2.3. The prohibition therefore applies to both providers and deployers of AI systems, each within their respective responsibilities, not to place on the market, put into service or use such AI systems. The remaining criteria for the prohibition on ‘social scoring’ are further described and analysed below.