AI incident response plan

Common AI incident scenarios

Prepare before anything happens

• Maintain an AI inventory with owners and credential locations — responders need it in the first hour.
• Document how to disable each agent and revoke each AI integration (the kill switch).
• Know each AI vendor's security contact and deletion-request process.
• Ensure AI activity logs exist and responders can access them.
• Define who decides on notifications (legal, privacy officer) and how to reach them.
• Create a simple, blameless reporting channel for employees who make a mistake.

Playbook: containment and investigation

Contain

1. Disable the agent or integration; revoke OAuth grants and rotate API keys it uses.
2. If data was entered into a vendor tool, preserve evidence (what, when, which account) and request deletion through the vendor's process.
3. Block further use of the affected tool or account if needed.

Investigate

1. Establish the timeline from AI tool logs, identity logs, and downstream system logs.
2. Determine what data was exposed or which actions were taken, and against which records.
3. Identify root cause: policy gap, excessive permissions, missing approval gate, injection, or credential leak.

Assess obligations

Work with counsel and your privacy officer to determine whether notification obligations apply — under HIPAA, state breach laws, contracts, or professional rules. Whether disclosure to an AI vendor constitutes a reportable breach depends on facts including the vendor's terms and agreements, so document them carefully.

Recover and improve

1. Restore affected systems or data.
2. Fix the root cause: reduce permissions, add approvals, update policy or training.
3. Re-assess the AI system and record evidence of the fix.

Run a tabletop exercise

Pick one scenario from the table and walk through it with IT, security, legal, communications, and the business owner in 60–90 minutes. Useful prompts: Who notices first? How do we know what the agent did? How quickly can we revoke its access? Who talks to the vendor? What do we tell the affected customer? Record gaps and assign owners.

Detection signals

Many AI incidents are discovered by accident. Build signals that surface them sooner:

• Employee self-reports through a blameless channel — often the fastest signal.
• Secret scanning alerts for AI provider keys.
• Unusual AI API spend or request volume.
• Agent actions outside normal patterns: new recipients, bulk operations, off-hours activity, permission errors.
• Customer or partner reports of unexpected messages.
• Vendor security notifications.

Roles and decision rights

Plan addendum template

Add an AI addendum to your existing plan with these headings: AI system inventory location; kill-switch procedures by system; AI vendor security contacts and deletion processes; AI-specific scenarios and first steps; log sources for AI activity; notification decision process; and post-incident review requirements, including re-assessment of the affected system.

Scenario walkthrough: misdirected agent email

A customer-support agent drafts and sends replies automatically. A customer reports receiving a reply containing another customer's order history.

1. First 15 minutes: pause the agent's send capability (or the whole agent) and revoke its mail-sending credential if pausing is not immediate.
2. First hour: pull the agent's action log to identify every message sent since the last known-good point; identify affected customers and the data involved.
3. First day: determine the root cause — for example, retrieval returning the wrong customer's records, or a prompt-injection attempt in an inbound message. Involve privacy counsel on notification obligations.
4. First week: fix the root cause, add approval for outbound messages containing account data, re-test, and document the incident and changes.

The walkthrough reveals whether you have the three things that matter most: an action log, a fast way to stop the agent, and a clear owner.