Oversight of Algorithmic Hiring Bias is a High Priority in California
Vendors claim that algorithm-driven tools are a better way to hire talent, but when these tools embed biases into employment practices, marginalized workers pay the price. On April 29, 2021, the California Department of Fair Employment and Housing Council held a hearing focused on this issue. CDT’s Lydia X. Z. Brown joined Upturn’s Aaron Rieke and Washington University Law Professor Pauline Kim to discuss ways to ensure that California’s antidiscrimination laws keep up with changing technology.
California is actively pursuing efforts to address algorithmic bias, leading the way for other jurisdictions.
By adopting these amendments, California rightfully put employers on notice that their algorithm-driven hiring tools are not exempt from antidiscrimination protections. New York City followed with its own attempt to better regulate private employers, proposing that employment decision technologies must be audited before sale for compliance with antidiscrimination laws.
Policy experts explain how hiring technologies embed bias and how to better protect workers.
Witnesses at the April 29 hearing explained how employment decision technologies work, and identified regulatory and enforcement gaps that must be filled to address discrimination. Upturn’s Aaron Rieke described how hiring technologies of varying levels of sophistication are used in different stages of the job application cycle – from job ads to the selection of candidates – without meaningful transparency or oversight. Professor Pauline Kim explained barriers to improving and regulating these technologies, including the problems of biased training data, uncertainty about the developer and vendor’s liability, and uncertainty about when potential job applicants count as “workers” who are protected by antidiscrimination laws.
CDT’s Lydia X. Z. Brown drew on CDT’s work examining the potential discriminatory effects of hiring technologies for job candidates with disabilities. They testified that the federal Americans with Disabilities Act makes disability discrimination illegal in ways that apply to algorithm-driven hiring, and California’s state law goes even further. California law clearly puts the onus on employers to make reasonable accommodations when they know of an applicant’s disability. And before employers implement discriminatory, albeit job-related, selection criteria in their hiring processes, they have an affirmative duty to search for alternative selection procedures or criteria that would not discriminate against disabled workers.
The California Department of Fair Employment and Housing (“Department”) enforces employment discrimination protections, while the Council issues regulations. CDT argued that, together, they can strengthen the state’s protections for disabled people. The Council’s regulations should require employers to:
Proactively investigate their tools’ design and identify all potential risks of disparate impact against all marginalized groups, including against disabled workers;
Only use the tools after proactively implementing potential accommodations, modifications, or alternative selection procedures that effectively mitigate each potential source of discrimination; and
Make audits of the tools’ design and impact easily available for public or government review.
Additionally, the Department and Council should educate the public and employers about how algorithm-driven hiring tools can discriminate, informing workers about how to recognize potential discrimination in the hiring process, and giving employers a chance to do right by workers. And when the Department learns that an employer is using a possibly discriminatory tool, it should use its authority to file Director’s complaints to independently investigate.
Policymakers must avoid allowing employers to default to minimal and ineffective auditing standards.
During the hearing, Council members were interested in understanding the effectiveness of self-regulation and quantitative auditing to reduce disparate impact. As Lydia explained, these steps alone are not effective accountability mechanisms for reducing discrimination:
Self-regulation by employers and vendors involves closed-door, opaque processes that keep workers at an information disadvantage.
Accountability requires transparency to the public about how tools are audited, and transparency to applicants about how tools work and will affect hiring decisions. But self-regulation allows employers and vendors to essentially vouch for themselves.
Quantitative auditing uses statistical analysis – an approach that excludes significant data about groups that are harder to compare.
As CDT has explained in our prior writing on bias auditing, statistical analysis cannot account for the diversity of disability and how it intersects with other marginalized identities. In contrast, qualitative auditing investigates the types and uses of collected data, the data’s job-relatedness, and the potential sources of risk to correct, and is far more preferable.
Policymakers should take note: the public is paying attention to algorithmic hiring bias.
Members of the public in the audience aired concerns during the hearing about additional barriers to algorithmic accountability: specifically, concerns about intellectual property and vendor liability. Citing intellectual property protection, employers and vendors have imposed nondisclosure agreements and impeded access to audit results so workers lack the information needed to legally challenge biased tools.
Additionally, open questions remain about whether companies that make hiring technologies can be held liable for resulting discrimination – for example, under theories that they have aided or abetted employment discrimination by an employer who uses their tool. These public concerns are well founded, and any regulation going forward should also consider the role these hurdles play in cementing a hiring system that perpetuates discrimination.
As algorithm-driven hiring tools and other hiring technologies become a mainstay of the employment landscape, the Council will need to modernize regulations to protect all marginalized workers. The public is counting on it.
As concern about risks and harms related to AI systems continue to grow, a growing chorus of policymakers, industry leaders, and advocates have called for independent AI assessments. This explainer provides an overview of recent proposals for third-party assessment in the United States, including state and federal legislation, executive actions, and industry proposals. In a separate post, we describe factors that will be required for third-party AI assessment to be effective.
Having third parties assess AI systems might seem like common sense, but crafting effective policies toward this goal can be devilishly tricky. A poorly-constructed ecosystem for third-party assessment could easily fail to consider the most consequential mechanisms of risk, neglect the AI harms that most impact people, or do more to protect AI companies than people.
Not All Guardrails Are Created Equal: Comparing Content Safety and Copyright Filtering
As courts and policymakers work through questions about chatbot liability, they should be wary of analogies that flatten meaningful technical differences. Copyright filtering and safety intervention share real challenges around ambiguity and evasion, but they diverge in what each control must assess, how each manifests over the course of a conversation, and how much can be verified from the outside.
Talking Tech with Roy Austin & Alex Givens on The Future of AI Governance
In this episode of CDT’s Tech Talks, Roy L. Austin, Jr., inaugural director of Howard University School of Law’s AI Initiative, joins Alexandra Givens, President and CEO of the Center for Democracy & Technology, to discuss the challenges and opportunities shaping the future of AI.
