How many categories can I define?

There is no hard limit, but classification accuracy is best with a small set of clearly distinct categories — typically two to ten. Adding a short description or example after each name sharpens the boundaries and reduces ambiguous calls.

What does the confidence score mean?

It is the model's self-reported certainty from 0 to 1 that the text belongs to the chosen category. Treat it as a relative signal rather than a calibrated probability, since LLM confidence is not statistically guaranteed.

What if the model does not return valid JSON?

The tool first tries to parse the response as JSON, and if that fails it falls back to matching one of your category names inside the raw text. You will see the raw output as reasoning so you can spot malformed responses.

Yes. Your key is used only for the direct request from your browser to OpenAI or Anthropic and is never sent to a Gera server or stored. You can revoke it anytime from your provider dashboard.

What is the LLM Output Classifier (BYO-key)?

Define your own categories with optional examples, then classify any text or LLM output with your OpenAI or Anthropic key. Returns the chosen category, a confidence score, and one-sentence reasoning. It runs free in your browser on Gera Tools, with nothing uploaded.

LLM Output Classifier (BYO-key)

Name: LLM Output Classifier (BYO-key)
Creator: Gera Tools
License: https://creativecommons.org/licenses/by/4.0/

Get one useful tool a week

Like this tool? Enter your email and we'll send you one genuinely useful Gera tool a week — plus a link to come back to this one. No spam, one-click unsubscribe any time.

Turn an LLM into a custom zero-shot classifier

You often need to sort free text — support tickets, model outputs, user feedback — into your own categories without training a model. This tool wraps a single API call in a structured classification prompt: you define the labels, paste the text, and get back one category, a confidence score, and a short justification you can review.

Zero-shot versus fine-tuned classification

Traditional text classification requires labeled training data and model training. Zero-shot classification with an LLM replaces that with a description of your categories in natural language. The tradeoff is speed and flexibility (define new categories instantly, no labeling pipeline) against consistency at high volume (where fine-tuning on labeled examples often wins). For internal tools, one-off analysis tasks, or categories that change frequently, zero-shot LLM classification is often the practical choice.

How to write good category definitions

The classification quality is proportional to how clearly you distinguish the categories. Compare:

Vague:

Bug
Feature request
Billing

Better:

Bug - user reports something broken or not working as expected
Feature request - user asks for new functionality or an enhancement
Billing - user has a question or issue about a charge, invoice, or subscription

Adding the dash-description takes ten seconds and typically makes a measurable difference in accuracy, especially at category boundaries.

How it works

You list your categories one per line, optionally adding a dash and a short description or example to disambiguate them. The tool builds a classification prompt that pins the model to choosing exactly one of your labels and asks it to respond as JSON with a category, a 0–1 confidence, and one-sentence reasoning. It then parses that JSON, normalizes the confidence (accepting either 0–1 or 0–100 formats), and falls back to a name-match if the model wraps the output in prose or code fences.

Tips and notes

Keep categories mutually exclusive — if the same input plausibly belongs to two categories, either merge them or add an exclusion note to each definition.
Low confidence usually means overlapping categories, not a hard example — refine the definitions before adjusting the model.
For high-volume use, switch to a cheaper model (gpt-4o-mini or claude-3-5-haiku) and validate a hand-labeled sample to check boundary accuracy.
Your key never leaves your browser except to call the provider directly, and it is never stored.

Use cases where this works well

This pattern — define categories as text, classify with a prompt — works particularly well for tasks that change frequently or where training data is hard to collect. Common uses include routing support tickets to the right team, tagging user feedback by theme, classifying model outputs for quality monitoring, and labeling search queries by intent. For tasks where the category set is stable and volume is high, consider building a labeled dataset and fine-tuning a smaller model to reduce ongoing API cost.