Chat template token overhead analyzer
Every chat API silently wraps your messages in template tokens — role headers, turn delimiters, sequence markers — and bills you for all of them. For long answers this is rounding error; for a high-volume classifier sending five-token messages, the wrapper can cost more than the content. This tool estimates that hidden overhead across four model families so you can see it in numbers.
How it works
The analyzer counts the tokens in your visible message content, then adds each model family’s per-turn template tokens:
- ChatML (OpenAI): a turn-start marker, a role header, and a turn-end marker per message.
- Llama 3: a begin-of-text token once, plus start/end header-id tokens and an end-of-turn token for each message.
- Mistral: instruction-wrapper tokens around user turns plus sequence BOS/EOS.
- Gemma / Gemini: start-of-turn and end-of-turn markers per message.
For each family it reports total template tokens, total prompt tokens, and the overhead percentage = template / (content + template). Content tokens are estimated from character count at roughly four characters per token.
Why the per-turn cost grows with message count, not just message length
Each additional turn in a conversation pays the template’s per-turn token tax regardless of how many words are in that turn. A single system prompt plus one user message and one assistant reply might be three turns; a ten-back-and-forth dialogue is twenty-three turns (1 system + 10 user + 10 assistant + the final BOS marker). The template cost scales linearly with turns, so a 200-token conversation of twenty one-token messages costs far more in template overhead than a 200-token single message.
This has concrete design implications:
- Consolidate short clarifications — rather than sending two short follow-up questions as separate turns, combine them into one user message.
- System prompt deduplication — if every call to a chatbot endpoint includes the same 500-token system prompt, and your provider charges for that system prompt on every call, prompt caching becomes high-leverage. Providers like Anthropic offer explicit prompt caching; OpenAI caches automatically for long repeated prefixes.
- Classifier and moderation use cases — if you are building a high-volume content moderation pipeline that sends short individual texts for classification, the per-turn template overhead may dominate. A non-chat completion endpoint or batched requests significantly reduce this overhead.
Template overhead by model family
The exact overhead varies by template, but the rough pattern is:
| Model family | Typical tokens per turn | Notes |
|---|---|---|
| OpenAI ChatML | 4–7 | Role header, start/end markers |
| Llama 3 | 6–10 | BOS once, plus header-id tokens per message |
| Mistral | 4–8 | Instruction wrapper, no explicit role headers |
| Gemma / Gemini | 4–6 | start-of-turn / end-of-turn markers |
For a ten-turn conversation, Llama 3’s higher per-turn cost adds roughly 60–100 tokens compared to 40–70 for ChatML — meaningful at scale but negligible for occasional calls. The percentages this tool reports tell you at a glance whether your conversation is template-light or template-heavy.
Tips and notes
- Overhead is dominated by turn count, not message length — fewer, larger turns are cheaper per token than many tiny ones.
- A repeated system prompt pays its content and template cost on every call unless your provider caches it; prompt caching is the single biggest lever for chat-heavy workloads.
- For extreme-volume short calls (moderation, classification), consider a non-chat completion endpoint to skip per-turn wrappers entirely.
- The percentages here are estimates; trust the API’s returned
usagefor billing, and use this tool to decide whether the overhead is worth optimizing. - Nothing you enter is sent to a server — all estimation runs locally in your browser.