Temperature & Top-p Explainer

Pick sampling settings with confidence

Temperature and top-p control how random an LLM’s output is, but the numbers are abstract until you see what they do. Drag the sliders to read a plain-English explanation of each setting in real time, or load a task preset and copy the recommended values straight into your API call.

How sampling works

After processing your prompt, the model produces a probability for every possible next token. Sampling parameters decide how that distribution is turned into an actual choice:

• Temperature rescales the probabilities. At 0 the model is nearly deterministic — it picks the most likely token almost every time. As temperature rises toward 1 and beyond, the distribution flattens, so less likely tokens get chosen more often. High temperature means more variety and more risk of going off-topic or producing factual errors.
• Top-p (nucleus sampling) caps which tokens are even eligible. It keeps the smallest group of top tokens whose probabilities sum to p, then samples only among those. A top_p of 0.9 trims the improbable tail; lowering it tightens the vocabulary the model can draw from.

The two parameters interact, so the standard advice is to tune one and leave the other at its default (temperature 1.0 or top-p 1.0 depending on which you are holding fixed).

A mental model for temperature

Think of temperature as the “boldness dial.” At temperature 0, the model would give the same answer every time for the same prompt — it always picks the highest probability token. At temperature 2, it might pick the fifth or tenth most likely token fairly often, producing surprising and sometimes incoherent results. The useful range for most tasks sits between 0 and 1.

For tasks with a single correct answer (extracting a date, writing SQL, answering a math question), low temperature keeps the model on the most probable, correct path. For tasks with many equally good answers (writing a marketing headline, brainstorming names, generating story ideas), higher temperature is useful because it explores more of the possibility space.

Task-based starting points

Common mistakes

Setting temperature to 0 for everything — deterministic output sounds safer, but for tasks like chatbots or writing assistance, it produces robotic, repetitive text that users find frustrating.

Cranking temperature and top-p both up — the combination amplifies randomness far beyond what either parameter alone would produce. The result is often incoherent. Tune one at a time.

Assuming temperature 1.0 is “creative mode” — temperature 1.0 is actually the model’s default, unscaled distribution. True creative tasks often benefit from values between 0.8 and 1.2 rather than extremes.

When in doubt, start at the preset, generate three to five samples, and nudge upward only if the outputs feel too repetitive or too similar to each other.