Why verify LLM output claim by claim
Long AI responses are dangerous precisely because they read smoothly. A single fabricated statistic or invented citation sits comfortably between ten correct sentences, and the fluent tone makes the whole block feel trustworthy. The fix is to stop reading the answer as one object and start treating it as a list of independent claims, each of which is either supported, contradicted, or unknown.
This tool does the mechanical part of that workflow: it breaks the response into individual claims and gives you a status column to fill in as you check each one.
Where hallucinations cluster
Not all sentences in an LLM response carry equal risk. These categories deserve special scrutiny when you work through the worksheet:
- Specific numbers and statistics. A model may correctly describe a trend while inventing the precise figure attached to it. “Adoption grew significantly” is often accurate; “adoption grew 47% in the first quarter” may be fabricated.
- Named citations. LLMs frequently cite plausible-sounding papers, reports, or books that do not exist. Always verify the source independently before treating it as real.
- Dates. Specific dates are easy to fabricate and hard to spot without checking. Events get placed in the wrong year, especially for anything near the model’s training cutoff.
- Proper nouns. Named individuals, organizations, and products attract confident hallucination. Job titles, roles, and affiliations are especially prone to being invented or swapped.
- Causal claims. “X caused Y” requires evidence. Models readily assert causation from correlation because the language pattern is common in training text.
How it works
Paste the response and the tool splits it into atomic claims by breaking on
sentence boundaries (., ?, !) as well as line breaks and bullet markers
(-, *, •, numbered lists). Each claim becomes a numbered row with a
True / False / Unsure dropdown. A running tally at the top shows how many claims
you have confirmed, refuted, or still need to investigate, so you can see at a
glance whether a response is mostly solid or riddled with unverified assertions.
When you are done, export the worksheet as a Markdown table to drop into a review doc, a pull-request comment, or an audit log.
Tips for an effective review
- Treat every number, name, date, and citation as a separate claim — these are where LLMs hallucinate most often.
- Default to Unsure, not True. A claim is only “True” once you have an external source, not because it sounds right.
- For research or legal use, keep the exported table alongside your sources so the verification is reproducible by someone else.
- If a single response has more than a handful of “False” rows, it is usually faster to regenerate it with a tighter prompt than to repair it piece by piece.
- Use the export as an audit trail when delivering AI-assisted research to a client or stakeholder — it shows which claims were verified and by whom.