Token count diff tool
When you tighten a prompt, the net token number tells you whether you won — but not where. This tool puts two versions side by side, reports the exact token delta and the cost change at your call volume, and shows a word-level diff so you can see precisely which sentences you cut and which you accidentally grew.
When to use this over a plain token counter
A plain token counter tells you the size of one version. The diff tool is for the moment after you edit a prompt: you need to know both whether the revision is smaller, and what changed, because prompt engineering has side effects. A cut you made to save tokens might have dropped a constraint the model needed. A clarification you added might have grown the prompt unexpectedly. The word-level diff surfaces both so you can be intentional rather than reactive.
How it works
Each version is tokenized with a character-based heuristic that closely matches tiktoken for English. The tool reports tokens for A, tokens for B, and the difference. It multiplies that difference by the chosen model’s input price per million tokens and your number of calls to give a cost delta. A word-level longest-common-subsequence diff then highlights additions in one color and removals in another. Everything runs locally.
The annualized cost of small deltas
A modest token saving on a high-traffic system prompt compounds quickly:
For example, saving 10 tokens per call at 1 million daily calls:
- Daily saving: 10,000,000 tokens
- At a hypothetical input price of $1 per million tokens: $10 saved per day
- Monthly: $300 — enough to matter in a cost-conscious deployment
This is why the diff tool pairs the delta with a cost calculation rather than just a number. The dollar figure makes the optimization concrete for budget discussions.
Reading the word-level diff
The diff highlights at the word level rather than the character level, which makes it easier to read for prompt engineering purposes:
- Added words (in version B but not A) are shown in one color — these are the phrases you inserted
- Removed words (in A but not B) are shown in another — these are what you cut
Look specifically for:
- Sentences that grew when you intended to shrink them (you rephrased verbosely)
- Constraints or instructions you dropped that the model still needs
- Boilerplate that crept back in from a paste
Tips
- The delta is more reliable than the absolute counts for cross-tokenizer comparisons — if both versions use the same style and language, the difference is accurate even when the totals drift slightly from a real tokenizer.
- Re-test model behavior after any significant cut. A leaner prompt changes the context, which can subtly shift output tone, format, or instruction-following.
- For very long system prompts, paste just the changed section to isolate the delta cleanly.