Find the right model for your context window
Picking an LLM starts with one hard constraint: does your prompt fit? This finder takes the minimum context window your task needs, filters every model down to the ones that support it, then sorts the survivors by price so you can see the cheapest model that can actually hold your data.
How to size your context window requirement
The context window is the total number of tokens a model can process in a single request — system prompt, retrieved documents, conversation history, and the output the model generates all share the same budget. To size your requirement, add up the largest realistic version of each component:
required = system_prompt + documents + history + expected_output + bufferA practical example: A document-summarisation task might have:
- System prompt: ~500 tokens
- A long document pasted in: ~15,000 tokens
- Conversation history (none, first call): 0 tokens
- Expected summary output: ~1,000 tokens
- 15% buffer: ~2,475 tokens
Total required: roughly 19,000 tokens. Any model with a 16K context window would fail; you need at least 20K and ideally 24K+ with comfortable headroom.
What the finder does
Enter your minimum context window requirement and the finder filters the model list to only those whose window equals or exceeds it. The resulting list is sorted by input price per million tokens so you can see the cheapest model that can actually hold your data. You can also set a volume to see an estimated monthly cost at your usage level.
Common context window scenarios and what they require
| Use case | Typical tokens needed | Why |
|---|---|---|
| Short Q&A or classification | Under 4K | Small prompt, short output |
| Single long document | 8K–32K | Depends on document length |
| Book or transcript analysis | 64K–200K | Full text must fit |
| Large codebase review | 100K–500K | Multiple files in context |
| Very long agent conversations | 64K+ | History accumulates |
Choosing between models that qualify
Once you have a list of qualifying models, the secondary selection criteria are:
Price — Input and output token prices differ significantly across providers. Use the per-call estimate to compare realistic monthly cost at your volume.
Output quality for your task — A 128K model may be available from a smaller provider at low cost, but if it underperforms on your specific task the price advantage is misleading. Benchmark on a sample.
Latency — Very large context windows can mean slower time-to-first-token, especially at high fill rates. If your application is latency-sensitive, test with representative prompt sizes.
Context degradation — Research shows LLM recall degrades on information buried in the middle of very long contexts (“lost in the middle” effect). Even if a model fits your data, verify it actually retrieves the relevant content reliably with a needle-in-a-haystack test at your typical load.
Tips
- Smallest window that fits, with headroom is almost always the best value — bigger windows cost more and recall degrades when you stuff them full.
- Output is the expensive half. A model with a huge window but pricey output tokens can cost more than a mid-size model for an output-heavy workload.
- Chunk and retrieve instead of paying for a giant window when only a small, relevant slice of your documents matters per request.