Human and Organizational Factors in AI Risk Management: Workshop 2 (Evaluation, Testing, and Oversight)
Date
Time
Location
Virtual
Time: 10:30AM – 4:30PM ET
Date: June 20, 2024
The National Academies of Sciences, Engineering, and Medicine will organize a workshop to identify and explore approaches to addressing human and organizational risks in AI systems. The emphasis will be on approaches that could be included in a more detailed guidance document complementing the recently issued “AI Risk Management Framework” from the National Institute of Standards and Technology (NIST). Areas of interest in the context of the NIST framework include human insights about AI-produced output and human oversight of AI systems and their operation in real-world environments.
A planning committee will be appointed to organize the workshop. The workshop will take the form of a series of sessions spaced over a couple of months in order to maximize opportunities for participation and allow time for participants to reflect on prior discussions. A proceedings in brief summarizing highlights of discussions across the sessions, will be prepared by a rapporteur (selected by the Academies and not a member of the planning committee).
This series of workshop sessions will identify and investigate human and organizational risk factors in AI systems. The hope of this work is to build off of the recently published “AI Risk Management Framework” from the National Institute of Standards and Technology (NIST) with a deeper exploration into human factors of AI risks.
“Evaluation, Testing, and Oversight” will be hosted virtually on June 20th starting at 10:30 am ET. The speakers will discuss how AI tool evaluations might be scoped, evaluation methods and mechanisms, and the governance of evaluations.
Speaker:
- Miranda Bogen, Director, CDT AI Governance Lab, Center for Technology & Democracy
