Validate NDJSON before you ingest it
The NDJSON Validator checks a newline-delimited JSON file line by line and tells you precisely which lines are broken. Run it before a BigQuery load or an Elasticsearch bulk request so a single malformed record does not fail or corrupt the whole ingest.
How it works
The input is split on newlines and each non-blank line is parsed with
JSON.parse. Lines that parse cleanly count toward the valid total. Lines that
throw are collected as errors, each recorded with its 1-based line number, the
parser’s own error message, and an 80-character preview of the offending line so
you can locate it immediately.
Strict mode adds one more rule: a line must parse to a JSON object, not a bare value, array, or null. This matches how NDJSON is normally used — one record object per line — and flags structurally valid but semantically wrong lines. Blank lines are skipped entirely, so a trailing newline never registers as a failure.
Why NDJSON breaks at ingest — and how to catch it before it does
Bulk data pipelines that accept NDJSON tend to fail silently or fail hard at the
wrong moment. BigQuery’s JSON load job stops on the first malformed line and
returns an error pointing to a byte offset, not a line number, making it slow to
debug. Elasticsearch’s _bulk API skips malformed action lines and returns
partial errors in a JSON response body that is easy to miss. Running this
validator first converts those cryptic failures into a simple list of line numbers
with the exact parse error message.
Common causes of malformed NDJSON
Trailing commas are the most frequent offender. A developer exports from a
JavaScript object and leaves { "id": 1, } — valid in some JS environments but
not in strict JSON. The parser reports an unexpected token at the comma.
Unquoted keys appear when output comes from a language that doesn’t always
quote object keys, or from a hand-edited file. { id: 1 } is a syntax error in
JSON even though it reads as valid in JavaScript.
Embedded newlines inside string values break the line-per-record structure.
A description field containing a literal newline splits one record across two
lines; the second fragment is not valid JSON on its own. The fix is to escape
the newline as \n inside the string.
Line-ending mismatches from Windows editors (CRLF) rarely cause parse
failures in modern parsers, but they can confuse byte-counting tools that expect
LF-only NDJSON. The validator splits on both \r\n and \n to handle this.
Strict mode and non-object lines — pipelines that expect records often break when they encounter a bare string or array line from an accidentally mis-formatted export. Strict mode catches these.
Tips and notes
The most common real-world error is a stray trailing comma or an unquoted key from a hand-edited file — the line number and preview point you straight at it. Turn strict mode off if your pipeline legitimately accepts non-object lines, for example a stream of bare numbers. Once the report shows all lines valid, the file is safe to stream into a line-oriented loader. For very large files, validate a representative sample first; if those pass, the structure is likely consistent.