Parse fixed-width records into a table
The Fixed-Width File Parser turns column-positional flat files — the kind produced by mainframes, banks, and legacy systems — into clean CSV or JSON. You describe each field by its name, where it starts, and how many characters it spans, and the tool slices every line accordingly.
How it works
You provide a spec with one line per field in the form name,start,length, where
start is the 1-based character position of the field’s first column. The parser
validates that each line has three parts and that start and length are positive
integers, reporting a precise error if not.
For every input line, each field is extracted with a substring from start - 1
for length characters. Because JavaScript’s substring stops at the end of the
string, lines shorter than the full record width yield empty strings for the
trailing fields instead of failing. With trimming enabled, each extracted value
has its surrounding whitespace removed — important because fixed-width fields are
space-padded to fill their column. The records are then serialised either as a
JSON array of objects or as quoted CSV.
Example and notes
For the spec:
id,1,4
name,5,16
city,21,9
the line 0001JOHN SMITH LONDON parses to id=0001, name=JOHN SMITH,
city=LONDON. Keep id as a string here — leading zeros matter, and CSV/JSON
text output preserves them. If your file uses a header line with column names but
no explicit widths, measure the start columns of each header label to build the
spec.
Where fixed-width files come from
Fixed-width (also called flat-file or column-positional) formats were dominant in the mainframe era when variable-length fields and delimiters were expensive to process. They survive today in several specific niches:
- Banking and financial systems: SWIFT MT messages, ACH payment files, and many bank statement exports use fixed-width layouts that have not changed in decades
- Government and tax data: IRS bulk data files, UK HMRC BACS submissions, and census microdata are commonly fixed-width
- Mainframe batch exports: IBM zSeries and AS/400 (IBM i) systems still output flat files from COBOL programs with fixed column widths
- Legacy ERP integrations: SAP, Oracle E-Business Suite, and older Dynamics versions produce fixed-width extracts for some interfaces
Interpreting a file without a spec
When no layout documentation exists, you can often recover the field structure by examining the data itself:
- Look for repeated character transitions: spaces that consistently appear at the same column in every row often mark field boundaries
- Find date and numeric patterns: dates like
20240101at a consistent offset confirm the field start - Check the file’s header row: if the first line contains labels but no data, the position of each label roughly indicates where the field begins
- Compare a few known records: if you know what one row represents, you can read off the column positions manually
A useful shell command for spotting column positions:
awk '{ for(i=1;i<=NF;i++) print i, $i }' OFS='\t' FS='' file.txt | head -80
This prints each character with its column number for the first record, revealing where fields likely start.
Output format comparison
CSV output: Best when the next step is Excel, Google Sheets, or a database import. Quoted CSV handles commas and line breaks within values. Leading zeros in numeric-looking fields are preserved as text when the CSV is opened in Excel only if the column is formatted as text before import.
JSON output: Best for piping into a programming environment, a REST API, or a data processing pipeline. Each record becomes a JSON object with the field names from your spec as keys. Leading zeros and spaces are preserved as-is in string values.