CDT Leads Letter to New York City Council on Pending Automated Employment Tools Bill

On February 16, 2021, the Center for Democracy and Technology sent an open letter cosigned by 20 local and national civil society organizations to New York City Council Majority Leader Laurie A. Cumbo regarding Introduction No. 1894.

The letter commends the Council for addressing the potentially discriminatory impact of automated employment decision tools while urging it to ensure that Intro. No. 1894 protects people with disabilities against the use of such tools in ways that may deny them economic opportunities.

The text of the letter is pasted below, but you can read the full letter as well as the list of signatories here.

***

Re: Amendments to Int. No. 1894 on the Sale of Automated Employment Decision Tools

Dear Majority Leader Cumbo,

We, the undersigned 20 civil society organizations, commend the Council for addressing the potentially discriminatory impact of automated employment decision tools in Int. No. 1894. We write to urge you to ensure that the bill protects people with disabilities against the use of such tools in ways that may deny them economic opportunities. Disabled people already face disproportionate hiring disparities, with an employment rate of about 34% for people with disabilities compared to 79% for people without disabilities. Because of biases in the data on which it is trained, AI can worsen these disparities.

As currently drafted, the bill requires “bias audits” to assess whether automated employment decision tools comply with New York City’s civil rights law and any other applicable employment discrimination laws. This broad requirement covers a wide range of protected classes, including disability. Int. No. 1894 also requires employers to inform applicants that they were evaluated through an automated employment decision tool, as well as the characteristics that the tool measured. This may alert applicants to the possibility that their disabilities may have affected the evaluation. The amended bill must preserve and build on these protections with disability in mind.

Automated employment decision tools that screen out disabled applicants may violate the Americans with Disabilities Act (ADA). Under the ADA, selection criteria must not “screen out, or tend to screen out” people with disabilities unless the criteria are “job-related” and “consistent with business necessity,” and applicants could not perform essential functions even with reasonable accommodations. To measure potential job performance, automated employment decision tools often derive selection criteria from characteristics of a model set of employees. These criteria may screen out applicants who do not share the same “desirable” characteristics, but who can perform the necessary job functions if they receive reasonable accommodations. This discrimination is difficult to discern when automated tools use facially neutral characteristics that disabled people may be unlikely to share. The Council must consider this issue when amending Int. No. 1894.

To better ensure that automated employment decision tools comply with employment discrimination laws and protect disabled people, the amended bill should address:

• Test design: Vendors must design their tools to comply with the ADA and any other applicable employment discrimination law. Statistical auditing cannot produce the same accuracy or breadth of data for disability as it may for race or gender, because it cannot capture the vast range of disabilities or people’s different experiences of the same disability. Vendors must describe how they design to account for bias that statistical audits will not capture, and demonstrate how their design improves the tools’ outcomes.
• Notice to applicants before evaluation: Employers must provide the notice required under Int. No. 1894 to applicants before an automated tool is used. The notice should explain the characteristics the tool measures and the mechanisms it uses to measure them, so that applicants know if they may require reasonable accommodations so they can use the tool.
• Opt-out and alternative tests: During evaluations, if applicants realize their disabilities are affecting how they engage with the tools, they should be able to opt out. Employers should not use automated tools without providing effective alternatives, including non-automated tests, to measure the same characteristics. Applicants’ evaluations should not be weighted negatively because they used alternatives instead of completing the test through automated tools.
• Notice to applicants after decision: Employers must also explain adverse hiring decisions and establish a process for applicants to obtain more information when the explanations are insufficient.

People with disabilities, especially those who are multiply marginalized, already experience disproportionate hiring disparities. In New York City, where government and corporate actions have an impact within and beyond city limits, there is an urgent need to improve economic equity across all identities. We urge you to incorporate the above changes to this bill so that automated employment decision tools do not deprive any protected class, including people with disabilities, of their right to economic opportunity

Read the full letter + the list of signatories here.