CDT Submits Comment on AISI’s Draft Guidance, “Managing Misuse Risk for Dual-Use Foundation Models”
CDT, in collaboration with Data & Society (D&S), submitted comments in response to the U.S. AI Safety Institute’s (AISI) request for comment on their draft guidance for developers on mitigating the risks associated with the misuse of foundation models. The guidance primarily addresses how developers can assess and reduce risks such as generating child sexual abuse material (CSAM) and non-consensual intimate imagery (NCII), enabling the development of chemical, biological, radiological, or nuclear weapons (CBRN), and facilitating cyberattacks and deception.
Our comments emphasize that the guidance does not adequately consider the sociotechnical context in which foundation models are deployed and may be misused. We urge AISI to broaden its scope to include risks related to bias and discrimination, as models misused in this way can also cause significant harm, particularly to marginalized and vulnerable communities. We also encourage AISI to recognize that the issues currently highlighted in the guidance are not independent of concerns around bias and discrimination. As one example, the generation of NCII is tied to existing patterns of gender-based online violence, making it critical for developers to account for gender bias in managing this type of misuse.
We also recommend that AISI offer more explicit guidance on how foundation model developers can involve subject matter experts, particularly social scientists, public health experts, and advocacy groups, in identifying, assessing, and mitigating misuse risks. Foundation model developers could consult these experts during risk assessments, include them in red teaming exercises, and involve them in interpreting results. This approach will lead to more comprehensive and contextually-aware risk management, improving the likelihood of addressing relevant risks.
Finally, given the inherent uncertainty in evaluating and mitigating misuse risks, we urge AISI to clarify how developers can establish reasonable risk tolerances, make release decisions based on them, and communicate their decision-making process transparently to stakeholders across the AI supply chain. Guidance on determining and communicating risk tolerance is especially crucial for open-weight models, where some risk mitigation strategies may not be feasible but where potential societal benefits may outweigh the associated risks.
