AI Policy & Governance, European Policy, Government Surveillance, Privacy & Data
Brief – Human Rights Risks of Facial Recognition AI Tech in Policing and Immigration Must be Properly Recognised in the EU AI Act
In April 2021, the European Commission unveiled its proposal for an Artificial Intelligence Act, which lays down harmonised rules on artificial intelligence that aim to ensure that Europeans can benefit from new technologies that are in compliance with fundamental rights. CDT Europe welcomed that step taken to protect society against risks related to the use of AI systems but also raised concerns about the lack of a rights-based approach in the draft Regulation, and the broad derogations to the prohibition on the use of remote biometric surveillance in publicly accessible spaces by law enforcement, which include untargeted facial recognition systems.
The EU AI Act, currently being negotiated in the European Parliament and the Council of the European Union, classifies uses of AI systems on a scale going from limited to high-risk, and prohibits the most dangerous applications. In doing so, the draft Regulation differentiates between remote real-time biometric identification techniques, such as voice recognition, retina and fingerprint scanning, and facial recognition systems, and ex post use of these techniques. Under the proposal, remote real-time use in publicly accessible places is prohibited, with, so many exceptions that they effectively swallow the rule. CDT previously explained that law enforcement’s use of facial recognition AI systems can pose a particularly high threat to human rights, given the risks of the improper deprivations of liberty that may result from such use, including racial profiling and indiscriminate surveillance . These risks are particularly acute when facial recognition is used in an untargeted manner—scanning a crowd looking for faces that match those in a database—as opposed to a targeted scan, such as where the face of a select individual is run through facial recognition in order to identify that person by matching them to a face in the database. As a result, CDT has generally called for a ban on law enforcement use of untargeted facial recognition and for moratoriums on law enforcement use of targeted use of facial recognition systems, until robust safeguards and effective limitations are in place.
This briefing paper aims to analyse how the EU AI Act addresses risks raised by the use of facial recognition systems by law enforcement and immigration authorities.
