Diff two AI responses
When you iterate on a prompt — change the system message, bump the temperature, upgrade the model — you need to know what actually changed in the output. Eyeballing two long responses misses subtle regressions. This tool runs a real diff between version A and version B and highlights every addition, deletion, and unchanged span.
How it works
The diff computes the longest common subsequence between the two token streams
(lines, or words within lines), exactly like git diff. Tokens that appear only in A
are deletions, tokens only in B are additions, and shared tokens are unchanged. The
result is rendered with colour coding so the differences jump out.
A: The capital is Paris.
B: The capital city is Paris.
-> The capital [+city] is Paris.When to use each diff mode
Line-level diff compares the text as whole lines. Use it when you want to spot structural changes: a paragraph that moved position, a bullet point that was added or removed, a section that disappeared. This is the right mode for comparing two summaries or two lists of recommendations.
Word-level diff breaks each line into individual words before computing the diff. Use it when you suspect small but consequential inline edits — a changed number (“reduced by 30%” vs “reduced by 20%”), a changed modifier (“reliably” vs “sometimes”), or a reworded key term. Word-level catches the edits that line-level hides as “line unchanged.”
For most prompt-iteration work, start with line-level to understand the structural changes, then switch to word-level for any line that looks superficially the same but reads differently.
Using the diff for prompt regression testing
A reliable pattern for iterating on prompts is to keep a golden output — a response from an earlier version of the prompt that you judged correct — as the A input. Each time you modify the prompt, paste the new response as B and run the diff. The highlighted additions and deletions show exactly what your change moved, which lets you verify that you improved the target behaviour without inadvertently degrading something else.
This is especially valuable when upgrading a model version. Two versions of the same model prompted identically can produce subtly different outputs. The diff reveals whether the behaviour your application depended on is preserved, rather than leaving you to re-read the full response and hope you notice the change.
Enable the trim whitespace option when comparing outputs from different platforms or copy-paste sources, since trailing spaces and blank lines at line boundaries can appear as false changes that obscure the real differences.