System prompt token auditor
Your system prompt is the most expensive text you own, because it rides along on every request. A bloated preamble that nobody re-reads still gets billed tens of millions of times a month. This auditor scans your prompt for filler, redundancy, and over-specification, estimates how many tokens you could shed, and turns that into a real annual dollar figure based on your own call volume and pricing.
Why the system prompt deserves special attention
Every input to an LLM API consists of the system prompt plus the conversation. The conversation varies per user and per turn. The system prompt is fixed — exactly the same bytes on every single call. That means any waste in the system prompt is not a one-off cost but a recurring charge that multiplies by your entire call volume, every day, indefinitely.
A prompt that was written in an afternoon and then iterated on over months accumulates phrases that felt important to add but were never audited for removal. “Please always remember to be helpful” adds zero behavioral signal over “be helpful” — yet it might have been added at some point and never questioned again. Multiply by 50,000 daily calls and that phrase has probably cost real money.
How it works
Token estimation: The tool uses a four-characters-per-token heuristic that tracks tiktoken closely for English prose (within a few percent for typical system prompts). For an exact count, use your provider’s tokenizer, but the heuristic is close enough to identify waste and plan savings.
Waste pattern scanning: The auditor identifies common categories of waste:
- Politeness padding: “Please make sure to”, “feel free to”, “don’t hesitate to”, “kindly ensure”. These phrases add tokens but convey no instruction that the model would not follow anyway.
- Hedging: “if possible, try to”, “ideally”, “where appropriate”. These soften instructions unnecessarily and sometimes produce inconsistent behavior.
- Redundant restatements: Instructions that appear in two different forms in the same prompt — the auditor detects bigram overlap and flags both occurrences, letting you decide which to keep.
- Over-specification: Long conditional chains that could be compressed: “If the user asks about X, do Y. If the user asks about Z, also do Y” collapses to “For any question about X or Z, do Y.”
Cost projection: You supply your daily call count and the input price per million tokens for your model. The auditor multiplies tokens saved per call by daily volume by 365 to show the annual saving. Even a modest-seeming per-call saving compounds significantly at scale.
Interpreting the output
The audit shows each flagged phrase or sentence with the category of waste and the token estimate for that segment. Review each flag rather than accepting all cuts automatically — some hedging is intentional (you want the model to use judgment in edge cases) and the auditor cannot know which.
The suggested slim version is a starting draft. Read it back as if you were seeing the prompt for the first time: does it still clearly specify the behavior you need?
Tips for the biggest wins
- Cut whole redundant sentences, not individual words. Two sentences that say the same thing cost twice as many tokens as one; picking the better phrasing and deleting the other is the highest-leverage edit.
- Move rare-case detail to the user turn. If a special instruction only applies when the user explicitly asks about a narrow topic, inject it in the user message at that point rather than carrying it in the system prompt on every call.
- After trimming, run your evaluation set. Shorter does not automatically mean better — occasionally a phrase the auditor flags as filler was actually providing a subtle behavioral cue. A regression costs more than the tokens saved.
- Pair with prompt caching. If your provider supports caching stable system prompts, a lean cached prompt is cheaper still than a lean uncached one. They are complementary optimizations, not alternatives.