AI incident severity classifier
When something goes wrong with an AI system — a harmful output at scale, a prompt that leaked data, a model returning another user’s information — the first job is triage: how bad is this, who needs to be paged, and how fast must we act? This classifier turns a short description of the incident into a P0–P4 severity level with the corresponding response timeline and playbook, so your first response is fast and consistent instead of ad hoc.
How it works
You describe the incident type (harmful/unsafe output, data exposure, security breach, availability/outage, or bias/discrimination), the number of affected users, and what kind of data was exposed, plus flags for regulatory-breach signals and ongoing/active status. The tool scores these factors — data sensitivity and breach indicators weigh heaviest, then blast radius, then reputational risk — and maps the total to a severity level. Each level comes with a target response window and a checklist: who to escalate to, containment, whether GDPR’s 72-hour breach-notification clock has started, and post-incident review.
What each severity level means and requires
P0 — Critical, active incident
The system is causing ongoing harm right now and cannot be allowed to continue. Immediate all-hands response. Typical examples: a public AI chatbot generating content that could cause imminent harm to vulnerable users, an active data exfiltration via a compromised model endpoint, or a bias failure affecting a live hiring or financial decisioning system at scale. Response: system taken offline immediately while the situation is assessed; leadership notified; legal and DPO engaged in the first hour.
P1 — High severity
The incident has caused or is likely to cause significant harm but is not necessarily ongoing in real time. Urgent response required within an hour. Examples: a prompt injection attack successfully exfiltrating other users’ data, a model producing unsafe medical advice that reached a large number of users, a data breach involving health or financial records of a meaningful number of individuals. The GDPR 72-hour notification clock is typically running at P1.
P2 — Moderate
A real problem that needs prompt attention but is not causing immediate harm at scale. Response window: within the business day. Examples: a model consistently returning biased recommendations in a lower-stakes context, a smaller-scale data exposure of non-sensitive data, a system-performance degradation affecting AI quality but not causing safety concerns.
P3 — Low
An isolated or edge-case issue with limited impact. Can be addressed in the normal sprint cadence. Typically a single-user complaint about a biased or inaccurate output that does not indicate a systemic problem, or a minor data-handling issue with no harm to individuals.
P4 — Informational / near-miss
No harm occurred but a scenario was detected that could have been serious under different conditions. Document, review, and use to improve monitoring and guardrails.
The factors that drive severity upward
Two factors push severity higher than blast radius alone would suggest:
Data sensitivity: Exposure of health, financial, authentication, or other special-category data triggers regulatory duties and potential serious harm to individuals, even if relatively few people are affected. A breach touching ten people’s medical records is more severe than one touching ten thousand people’s email preferences.
Active and ongoing status: An incident that is still happening as you triage is more severe than one that has already stopped, because every minute of delay causes additional harm. The “is this still active?” flag is one of the first questions the classifier asks.
Notes and use
Severity can escalate as facts emerge, so re-run the classifier when the blast radius becomes clearer. An incident that looks like P3 often becomes P1 once you discover the data was sensitive or the exposure was wider than initially understood. This is a triage aid: legal counsel and your DPO own the final call on regulatory notification decisions, and your incident runbook owns the named on-call rotation. Use it to get the right people engaged on the right clock from minute one.