Regex generation prompt builder
LLMs are notoriously confident and wrong about regular expressions — they hand back patterns that look plausible but quietly fail on the third example. The fix is test-driven prompting: give the model concrete strings that must match and must not match, and require it to prove the pattern against them. This builder assembles exactly that prompt, plus requests for an explanation and a list of edge cases.
How it works
You describe the pattern, pick a regex flavor, and list must-match and must-not-match strings. The tool builds a prompt that asks the model to return four sections: the pattern in idiomatic syntax for your flavor, a token-by-token explanation including every capture group, a test table that confirms the expected result for each of your strings (with instructions to fix and re-table if any fail), and at least three edge cases or limitations. It also warns the model away from features your flavor does not support.
Why LLM regex generation goes wrong without test strings
A language model generating a regex faces a specific problem: it has read millions of regex examples in training, so it produces patterns that look familiar and plausible, not patterns that are correct for your specific inputs. Common failure modes:
- Anchoring mismatch. The model writes
^\d{4}-\d{2}-\d{2}$but your input has the date embedded in a larger string, so it never matches. - Wrong character class.
\dmatches only ASCII digits in most JavaScript engines but all Unicode decimal digits in Python’sre. The model may not pick the right behavior for your flavor. - Greedy vs. lazy quantifiers.
.*and.*?look similar but match very differently in multiline strings. Models default to greedy without thinking. - Catastrophic backtracking. Some patterns that look reasonable have exponential worst-case behavior on certain inputs. The edge-case section in the generated prompt asks the model to flag this risk.
Providing must-match and must-not-match strings converts the task to one the model can verify against, which surfaces most of these problems in the output itself.
Regex flavor differences that bite most often
| Feature | JavaScript | Python re | PCRE | Go RE2 |
|---|---|---|---|---|
Lookahead (?=...) | Yes | Yes | Yes | No |
Lookbehind (?<=...) | Limited | Yes | Yes | No |
Named groups (?P<name>...) | ES2018+ | Yes | Yes | Yes |
\d matches Unicode digits | No | Yes | Yes | No |
| Non-possessive quantifiers only | No | No | No | Yes |
Picking the wrong flavor and getting a pattern that uses an unsupported feature is a silent failure — the engine either errors or silently falls back to different behavior. Declaring the flavor in the prompt prevents this.
Tips and notes
- More test strings, fewer surprises. Add the awkward real-world inputs you actually expect — that is where naive patterns break.
- Pick the right flavor. Go’s RE2 has no lookahead or backreferences; declaring the flavor stops the model from using them.
- Read the edge-case section. It often reveals an assumption (like anchoring) you will want to change before the pattern goes into production.
- Always test for real. Test-driven prompting reduces errors but does not eliminate them — validate against production data.
- For catastrophic-backtracking risk, ask the model to suggest a linear-time alternative; most pathological patterns have one.