Artificial Intelligence System Risk Management (Governance) Survey Example

1. Introduction

This survey contains ten example questions about governance of "designing, developing, deploying, or using AI systems", based on the Artificial Intelligence Risk Management Framework ("AI RMF 1.0").¹ The primary source for the questions is the AI RMF Playbook ("Playbook"), which states that:

"[This Playbook] provides suggested actions for achieving the outcomes laid out in the AI RMF. ... Suggestions are aligned to each sub-category within the four [AI RMF 1.0] functions (Govern, Map, Measure, Manage)."²The examples shown here represent a selection of questions about Govern controls.

Questions were generated using a ChatGPT 4.0 Team dedicated workspace. The tool was asked to translate text from the Playbook into two forms of control questions that it had been previously "taught":

Multiple choice, single answer questions ("Simple Questions"), and
Multiple-choice, multiple-answer questions ("Complex Questions").

DS&A checked the generated questions and provided further prompts to ChatGPT to refine its translation (e.g., directing the tool to use more closed-end questions, or use a Simple Question to ask if the Bank had implemented a Playbook-recommended control, followed by a Complex Question to ask about two or more recommended component features of the control).³

Thank you.
John
John Seddon
john@firiskassessment.com
310 344 2408

¹Artificial Intelligence Risk Management Framework, NIST AI 100-1, NIST, January 2023.
² AI RMF Playbook, NIST, January 2023.
³ Four types of human-AI collaboration are described in Navigating the Jagged Technological Frontier: Field Experimental Evidence of the Effects of AI on Knowledge Worker Productivity and Quality, Working Paper 24-013, Harvard Business School, September 22, 2023.