AI decision newspaper test
The newspaper test is an old ethics gut-check: would you be comfortable seeing this decision on tomorrow’s front page? For AI systems, a single output can become that story — and it can go wrong in two opposite directions. This tool runs your decision through both: the over-harm angle, where a dangerous or biased output makes headlines, and the over-caution angle, where an absurd or harmful refusal makes headlines just as easily.
How it works
You describe what your AI system decides or outputs and the context it operates in, then score the factors that drive each kind of headline — for over-harm, the severity of a bad output, the vulnerability of affected people, and how irreversible the consequence is; for over-caution, how important the refused request might be and how visibly absurd a refusal would look. The tool computes a risk level in each direction and generates a plausible illustrative headline for both, with the contributing factors listed so you know exactly what to mitigate before launch.
The dual-direction problem most teams miss
Early AI safety thinking focused almost entirely on the over-harm direction: prevent the system from saying something dangerous, biased, or offensive. The result, in many products, was systems that refused reasonable requests at a high rate — declining to give first-aid instructions, refusing medical information to patients, blocking legitimate financial queries. Those refusals became news stories too.
The test scores both directions because both represent real product failure. An AI safety review that only asks “could this output cause harm?” and never asks “could this refusal cause harm, or look absurd?” is optimising for only one type of front page. Designing for the middle — genuinely helpful and genuinely safe — requires holding both failure modes in view simultaneously.
How to apply the generated headlines
The over-harm and over-caution headlines the tool produces are not predictions. They are prompts for imagination: plausible stories that make the abstract risk concrete. The most effective use of them is in a pre-launch review meeting. Read the worst-case headline aloud and ask the team whether the current system could produce the scenario described. That conversation surfaces mitigations that do not emerge from abstract risk scoring.
If the over-harm headline is plausible, look at the severity and irreversibility factors — those drive the most damaging stories. If the over-caution headline is plausible, look at how important the refused use case is and whether the refusal would look unreasonable to a mainstream observer.
Tips for the scoring
- Vulnerability raises the stakes. The same output is a bigger story when it affects children, patients, or people in crisis — weight those use cases up.
- Irreversibility is the multiplier. A wrong answer you can correct is recoverable; a wrong automated decision that denies a loan or a job is the front-page version.
- High visibility amplifies both directions. Consumer-facing AI in a public product gets far more scrutiny than an internal tool — adjust accordingly.
- Use the test iteratively. Run it before launch, and again whenever a major behaviour change ships.