A fact-check prompt builder turns a vague “is this true?” into a disciplined evaluation an LLM can follow. Left to themselves, models tend to either rubber-stamp a claim or confidently reject it, often inventing supporting “evidence” along the way. This tool builds a prompt that forces the model to separate evidence from counter-evidence, name its sources, state caveats, and refuse to guess when it cannot verify — then optionally runs that prompt with your own API key.
How it works
You paste the claim exactly as stated, add context (date, region, source, domain) so the model evaluates the right version, and pick a required confidence level. The builder assembles an instruction block telling the model to list supporting evidence, contradicting evidence, key assumptions, and a final verdict on a fixed scale — and, crucially, to return “unverifiable” rather than fabricate facts when it lacks grounds. If you supply an OpenAI or Anthropic key, the prompt runs directly from your browser against the provider’s REST API; otherwise you copy it into any chat model. Nothing is sent to our servers, and your key is used only for the single request.
What the output looks like
The structured prompt instructs the model to return its evaluation in four sections:
- Supporting evidence — what the model knows that is consistent with the claim being true, with source names where possible.
- Contradicting evidence — anything that challenges or qualifies the claim.
- Key assumptions and caveats — what would need to be true for the claim to hold, and where uncertainty lies.
- Verdict — one of: Supported, Partially supported, Contested, Unlikely, or Unverifiable. The model is instructed to choose Unverifiable rather than guess when it cannot provide grounded evidence.
This structure makes the model’s reasoning transparent and easier for a human reviewer to challenge.
Choosing the right confidence level
| Setting | When to use it |
|---|---|
| Standard | General fact-checking, background research, editorial prep |
| Strict | Medical, legal, financial claims; anything with real-world consequences |
| High | High-stakes publishing; requires the model to name checkable sources and flag training-data uncertainty explicitly |
At strict and high levels, the prompt instructs the model to decline giving a verdict on recent events that may postdate its training, which reduces confident-but-wrong outputs.
Tips and examples
For high-stakes claims (medical, legal, financial), set strictness to high and ask the model to cite specific, checkable sources — then verify those sources yourself, since models can cite plausible-sounding references that do not exist. Always provide context: “GDP grew 2%” needs a country and year before it can be judged. Use the counter-evidence section as your checklist — if the model cannot surface any opposing view, that often signals it is pattern-matching rather than reasoning. For live or recent claims, pair this with a model that has web search, because a base LLM only knows its training cutoff.
A common mistake is to check the verdict without reading the caveats. A “Partially supported” verdict with a caveat that says “this was true before the policy was reversed” tells a very different story from a clean “Supported” — and both could look like a green light if you skip the caveats section.