System prompt linter
A system prompt is the most leveraged text in your application — every response inherits its strengths and its bugs. This linter runs a battery of heuristic checks for the problems that quietly degrade prompt quality: no output format, vague instructions, contradictory rules, missing scope boundaries, and bloat. You get a scored report with specific, actionable fixes, updated live as you edit.
Why linting matters before you test
Most prompt issues only surface during real use — an edge case the developer did not anticipate, a user query that exposes an ambiguity. By that point, you have a deployed application with a hard-to-diagnose inconsistency. Running the linter before you deploy catches the structural issues that reliably produce poor responses, without requiring you to enumerate every edge case in advance.
Think of it the same way you think of a code linter: it does not prove your program is correct, but it catches a class of easy mistakes before they waste your time.
How it works
The linter scans your prompt text against a fixed rule set, each targeting a known anti-pattern:
Output format check: Does the prompt say how the answer should be shaped — plain text, JSON, a bullet list, a specific schema? Absent an explicit format instruction, most models default to prose, and the output structure will vary unpredictably. This is the single most common cause of inconsistent responses in production.
Specificity check: Does it lean on vague verbs like “help”, “handle”, “assist”, or “be good at” without a concrete instruction? “Help users with billing questions” is worse than “Answer billing questions about invoice status, payment methods, and refund eligibility. Escalate account cancellations to the human queue.”
Conflict detection: Does the prompt simultaneously demand brevity and completeness? Tell the model to be concise and comprehensive? Be formal but conversational? Conflicting constraints produce erratic behavior because the model has to pick one on each response — and it will pick inconsistently.
Scope and refusal check: Does the prompt define what is out of bounds and what the model should say when a request falls outside its scope? Without this, models will confidently attempt to answer things they should not — or worse, they will silently give a poor answer rather than explaining they cannot help with that.
Length check: Very long prompts dilute attention and increase the chance of conflicting rules being added over time. The check is a soft warning, not a hard limit, but prompts over a threshold are flagged for review.
Filler check: Does the prompt contain politeness padding and meta-instructions that consume tokens without conveying information?
Each rule returns pass, warn, or fail with a one-line reason. The overall score reflects how many checks clear. Everything runs in your browser — instant and private.
Common fix patterns
| Issue | Typical cause | Fix |
|---|---|---|
| No output format | First draft of prompt | Add “Respond in JSON with keys: …” or “Use plain sentences, no headers.” |
| Vague verbs | Instructions written casually | Replace “help with X” → “answer questions about X; if the user asks Y, do Z.” |
| Conflict | Two authors edited separately | Pick one constraint per dimension; explain when exceptions apply. |
| No refusal clause | Assumed the model would “just know” | Add “If the user asks about [out-of-scope topic], say [specific response].” |
| Excessive filler | Polite drafting style | Delete every sentence that starts with “Please note that”, “Keep in mind”, “It is important”. |
Tips
- Treat warnings as nudges and failures as bugs. Fix failures before shipping; address warnings in the next revision.
- A clean linter score is a green light to start testing — not a substitute for it. The linter catches structural problems; you still need to test behavior on real inputs.
- Re-run the linter after each major edit to catch regressions introduced while fixing something else.