Report – Regulating Robo-Bosses: Surveying the Civil Rights Policy Landscape for Automated Employment Decision Systems
Introduction
In December 2023, the Center for Democracy & Technology (CDT), in collaboration with a broad range of national civil rights and workers’ rights organizations, published the Civil Rights Standards for 21st Century Employment Selection Procedures (the “Civil Rights Standards” or simply the “Standards”), a detailed set of policy recommendations regarding the methods and tools that today’s employers use to recruit and assess workers.[1] The key impetus for the Civil Rights Standards was employers’ increasing use of automated employment decision systems (AEDSs) to evaluate employees and make employment decisions.
The rise of AEDSs underscores the degree to which antidiscrimination regulation has failed to keep pace with companies’ recruitment, hiring, and personnel management practices in recent decades. Workers subjected to AEDSs are at an extreme information disadvantage, with little insight into how they are assessed or whether they face a risk of an unfair or discriminatory decision. This is deeply concerning because there is scant evidence that AEDSs are more effective than simpler and more transparent employment assessments, but considerable evidence that AEDSs can discriminate against candidates from protected groups.[2]
The Standards sought to provide advocates, policymakers, and workers alike with a roadmap on how to address these risks. The Standards made policy recommendations in five categories:[3]
- Notice and explanation: Require companies to provide concise disclosures to candidates about the key features of any AEDS they use, publish detailed summaries of all AEDS audits, and maintain records to ensure relevant materials are available if an AEDS leads to discrimination.
- Auditing: Ensure that independent auditors test AEDSs for both discrimination risks and job-relatedness both before deployment and at least annually thereafter.
- Nondiscrimination: Require employers and vendors to take proactive steps to minimize potential causes of discriminatory outcomes in their selection tools, use the least discriminatory tools available, explore accommodations and more accessible alternative selection methods, and refrain from using certain tools that pose a particularly high risk of discrimination.
- Job-relatedness: Require companies to conduct detailed validity studies to ensure a selection procedure is the least discriminatory valid method of measuring a candidate’s ability to perform essential job functions.
- Oversight and accountability: Allow candidates to raise concerns about a selection procedure, appeal its results, or opt out of its use altogether; and ensure robust enforcement for discriminatory AEDSs by making vendors and employers jointly responsible for resulting harms.
These recommendations provided “a concrete alternative to recent proposals that would set very weak notice, audit, and fairness standards for automated tools.”
The pace of AEDS legislation and policy proposals continued to increase in 2023 and into 2024. Nationally, at least eleven bills were pending at the end of 2023 purporting to target AEDS-driven discrimination. At least seven more bills across six states followed in the first weeks of 2024.
Although the increased legislative attention to AEDSs is a welcome development, much of the proposed legislation falls short of what is needed to address the risks that AEDSs pose. This report surveys the current policy landscape in the year-plus since the Standards’ publication by analyzing legislation introduced or enacted in the subsequent months. Its goal is to help policymakers and advocates understand the structural approaches to AEDS regulation embodied by current legislation and evaluate how they do and do not incorporate the Standards’ recommendations. That evaluation, in turn, provides a roadmap for needed improvements in legislation to help prevent AEDSs from giving rise to increased discrimination in employment decisions.
[1] Civil Rights Standards for 21st Century Employment Selection Procedures (2022), https://cdt.org/wp-content/uploads/2022/12/updated-2022-12-05-Civil-Rights-Standards-for-21st-Century-Employment-Selection-Procedures.pdf (Civil Rights Standards). [https://perma.cc/D26W-LZNV]
[2] See generally, e.g., Hilke Schellmann, The Algorithm: How AI Decides Who Gets Hired, Monitored, Promoted, and Fired and Why We Need to Fight Back Now (2024); Olga Akselrod & Cody Venzke, How Artificial Intelligence Might Prevent You From Getting Hired, ACLU, Aug. 23, 2023, https://www.aclu.org/news/racial-justice/how-artificial-intelligence-might-prevent-you-from-getting-hired [https://perma.cc/MP65-AZDN]; Lydia X.Z. Brown, et al., Algorithm-driven Hiring Tools: Innovative Recruitment or Expedited Disability Discrimination?, CDT, Dec. 3, 2020, https://cdt.org/insights/report-algorithm-driven-hiring-tools-innovative-recruitment-or-expedited-disability-discrimination/ [https://perma.cc/MBW7-YJC6]; Jeffrey Dastin, Insight – Amazon scraps secret AI recruiting tool that showed bias against women, Reuters, October 10, 2018, https://www.reuters.com/article/idUSKCN1MK0AG/. [https://perma.cc/EHR9-TD3Y]
[3] These categories come from the Civil Rights Principles for Hiring Assessment Technologies (Civil Rights Principles), which The Leadership Conference on Civil and Human Rights published in 2020 with input and endorsements from CDT and more than 20 other civil rights and workers’ rights organizations. Civil Rights Principles for Hiring Assessment Technologies (2020), https://civilrights.org/resource/civil-rights-principles-for-hiring-assessment-technologies/. [https://perma.cc/Q2LC-WPXE]
