Incident Response: Preparing for AI-Caused Accidents
Risk management prepares "on the premise accidents happen." Have procedures ready for AI-caused incidents (info leakage, wrong output, runaway).
Assumed Incidents
- Inputting confidential/personal info into external AI
- Putting AI's wrong output straight to a customer
- Agent/integration runaway or misoperation
- Information leakage via prompt injection