Keep long conversations inside the context window
Long-running chats eventually overflow the model’s context window — or waste tokens replaying stale history that no longer matters. This tool builds a rolling summary: it keeps your most recent turns verbatim and asks the model to compress the older portion into a dense factual block, so the whole conversation fits your token budget without losing the facts, names, decisions, and open questions that matter.
How it works
You paste your conversation as a JSON array of { "role", "content" } objects and set a target token budget. The tool divides that budget into two zones:
- Recency window (~40%) — the newest turns are passed through unchanged, because recent context carries pronoun references and immediate intent that should not be paraphrased away.
- Compressed history (~60% of budget ceiling) — all older turns are sent to your chosen model (OpenAI or Anthropic) with a prompt that asks for a compact summary preserving every named fact, decision, commitment, and unresolved question.
The output is a single context block — the summary first, then the verbatim recent turns — ready to paste into the next call as the new conversation history. Token estimates use the ~4 characters-per-token heuristic for speed.
When to use this
This pattern is most useful when:
- A multi-day support or research chat has grown past 10,000 tokens and starts hitting context limits.
- You want to archive a conversation but carry its key facts into a fresh session.
- You are building a persistent AI agent that must carry state across many API calls without re-sending the full transcript every time.
Practical tips
Leave headroom. Set the budget a little below your actual context window limit so there is room for the next user message and the model’s reply. A good rule of thumb is 80–85% of the window.
Check for dropped facts. If the summary misses a detail you need, either raise the budget so the recency window is wider, or move that specific turn out of the compressible portion before running.
Iterative compression. For very long histories, compress in stages: run compression once, treat the output as the new history, then compress again if needed.
API key safety. Your key is sent only in direct browser-to-provider requests and is never stored or routed through any server. You can revoke it at any time from your provider’s dashboard.