Evidence-based response prompt builder
Most prompts ask a model for an answer and accept whatever comes back at face value. The problem is that language models flatten the difference between a strong source and a weak one — a forum post and a systematic review get the same confident tone. This builder writes a prompt that forces the model to slow down: enumerate its evidence, label each item by strength, and let the weight of the evidence drive the conclusion rather than the other way around.
The problem this solves
A language model answering a research question from its training data behaves like a confident generalist who has read everything but tracks none of it. The answer you receive may draw on a single blog post, a clinical trial, and a decades-old textbook equally — without any signal about which is driving the conclusion.
Evidence grading is the practice, borrowed from clinical medicine and systematic review methodology, of labelling sources by how much weight they deserve. A meta-analysis of randomised controlled trials sits at the top; an expert’s opinion or a single case report sits at the bottom. The key benefit is not that grading makes citations accurate (models can still hallucinate references) but that it makes the reasoning visible: you can see what tier of evidence the conclusion is built on, and whether that tier is strong enough to act on.
How it works
The generated prompt has three enforced phases. First the model lists every piece of evidence relevant to your question. Second, it tags each item against the evidence strength scale you pick — for example anecdotal, observational study, randomized trial, or meta-analysis. Third, it writes a conclusion that explicitly references the strongest available evidence and flags where the evidence is thin or conflicting. Citation and conclusion formats are configurable so the output drops cleanly into a report or a literature note.
Evidence scales available
Clinical / scientific scale (adapted from evidence-based medicine hierarchy):
- Meta-analysis or systematic review
- Randomised controlled trial (RCT)
- Cohort or case-control study
- Cross-sectional study
- Case series or case report
- Expert opinion or editorial
- Anecdotal or informal observation
General knowledge scale (for non-academic questions):
- Scientific consensus / established theory
- Peer-reviewed study
- Government or institutional report
- Expert opinion from named source
- Journalism or secondary reporting
- Anecdote or personal account
Use the clinical scale for medical, nutritional, and scientific questions where the RCT hierarchy applies. Use the general scale for policy, business, or social questions where the academic hierarchy does not map cleanly.
Tips for better results
Supply your own sources when you can. Paste documents below the prompt so the model grades real evidence rather than its training memory, which substantially reduces fabricated citations.
Watch for forced confidence. If the model still concludes strongly from weak evidence, add the instruction: “If all available evidence is at tier 5 or below on the scale, state that explicitly and decline to make a strong claim.”
Use the strength labels as a filter. When reviewing the output, read the top-tier rows first — they carry the most weight and immediately reveal whether the conclusion is genuinely supported or mostly constructed from lower-quality sources.
Treat citations as hypotheses, not facts. Even with grading instructions, a model can invent plausible-sounding citations. Use the output to identify what evidence would be decisive, then verify those specific sources independently.