Large language models sometimes produce text that reads fine but is quietly repetitive — the same connectors, the same hedges, the same favourite adjectives over and over. This analyzer gives you the numbers behind that gut feeling: word frequencies plus three diversity metrics computed entirely in your browser.
What this reveals about LLM output specifically
Word frequency analysis surfaces problems that are invisible to a quick read but obvious in the numbers. Common LLM patterns the metrics catch:
Template lock. A model producing multiple outputs from the same prompt often reuses a small set of structure words (“furthermore,” “notably,” “it is important to note”) across every response. A frequency table dominated by these phrases signals the model is drawing from a template rather than generating fresh prose.
Topic drift. If a content word you expected to dominate the frequency table is absent or low, the model may have drifted off the actual topic. For example, a brief on renewable energy that ranks “technology” and “future” highly but barely mentions “solar” or “wind” has drifted toward generalities.
Padding and hedging. A very low lexical density — most words are function words and hedges — is a diagnostic sign of verbose, content-light output. “It is worth noting that, in many cases, it may be possible to consider…” is high in function words and low in content words.
How it works
The tool tokenizes your text into words (lowercased, stripped of punctuation), then counts how often each one appears. With the stopword filter on, ultra-common function words are removed so the ranking reflects real subject matter. From the counts it derives three figures: unique word count, type-token ratio (unique words ÷ total words), and lexical density (content words ÷ total words). The top-N table shows the most frequent words with their counts and share of the text.
Reading the metrics
- A high type-token ratio means varied vocabulary; a low one means the text leans on a small set of words.
- Lexical density below roughly 40% often signals padding, filler, or boilerplate phrasing.
- A frequency table where one or two words dwarf the rest is a classic sign of a model stuck in a loop or over-using a template phrase.
Tips
- TTR shrinks naturally as text grows, so only compare passages of similar length.
- Run two model outputs for the same prompt through the tool to see which one writes with more range.
- If a content word you expected to dominate is missing from the top of the list, the model may have drifted off topic.
- Toggle stopwords off to see exactly which function words are inflating the count — “actually” and “essentially” appearing repeatedly are common LLM filler tells.