Visualize your token budget before you hit the limit
Every model has a fixed context window — the total number of tokens it can read and write in a single call. That budget is shared across your system prompt, any retrieved context, the conversation history, the user’s question, and the model’s own reply. This tool lays all of those out on a single stacked bar so you can see at a glance whether you fit, and which section is eating the most space.
How it works
Pick your model to set the total token limit, then paste each part of your prompt into its own box. The tool estimates token counts using a character-based heuristic (about four characters per token for English), adds the output tokens you want to reserve for the reply, and renders a color-coded bar showing the share of the window each part consumes. If the total exceeds the limit, the bar turns red and the overflow amount is shown.
Why each section gets its own color
The visualizer assigns a distinct color to each component: system prompt, retrieved context chunks, conversation history, user message, and reserved output. This is not just aesthetic — it tells you at a glance which component is the dominant consumer, which is the most likely culprit when you approach overflow, and which is fixed versus controllable.
The system prompt is typically fixed for a given application; it is the floor on your budget. Retrieved chunks are the largest variable component and the first lever when you need to trim. Conversation history grows with every turn and is the sneaky accumulator that causes overflow in long sessions even when individual turns are short. The reserved output block is easy to forget — if you leave it at zero, you are implicitly leaving no room for a reply.
A common architecture walkthrough
For a RAG application answering questions over a knowledge base:
- System prompt — 300 tokens. Fixed. Contains instructions, output format, and persona. Goes first and does not change.
- Retrieved chunks — 3 chunks at roughly 400 tokens each = 1,200 tokens. Variable per query. These are the paragraphs most relevant to the user’s question.
- Conversation history — 600 tokens. Grows per turn. After 5-6 exchanges, this starts to crowd the chunks.
- User message — 50 tokens. Almost always short.
- Reserved output — 800 tokens. Enough for a detailed multi-paragraph answer.
Total: 2,950 tokens. On a 4,096-token model, this barely fits. On a 32k model, it is negligible and you can retrieve 10-20 chunks instead of 3, dramatically improving answer quality.
Paste these sections into the visualizer to confirm the actual numbers before building the application — what looks comfortable on paper sometimes overflows in practice once you add longer queries or richer history.
Tips for managing the budget
- Reserve output tokens first. Allocate what you need for a good response before fitting inputs, not as an afterthought. If you regularly need 2,000 token replies, budget for them from the start.
- Rank chunks and fit as many as the budget allows. Score retrieved chunks by relevance, then fit them in score order until the budget is exhausted. The last chunk admitted is the least relevant one the model sees.
- Summarize conversation history at a threshold. Once history exceeds, for example, 30% of the window, compress older turns into a summary block. This keeps recency and saves space without dropping context entirely.
- Use the 10-15% safety margin. Because the four-characters-per-token heuristic is approximate, stay comfortably below the hard limit — not within 50 tokens of it.
If you consistently overflow, summarize old history or move to a larger-context model rather than blindly dropping chunks.