Clean up a messy CSV export
The CSV Cleaner fixes the small inconsistencies that break CSV imports: invisible byte-order marks, mixed line endings, stray whitespace around values, and blank rows. It parses your file with a quote-aware reader, applies the fixes you choose, and re-emits clean, normalized CSV with a before/after summary.
How it works
First, if BOM removal is on and the file begins with U+FEFF, that character is
stripped. The cleaner counts CRLF (\r\n) sequences in the raw text so it can
report how many line endings are normalized — the output is always written with
LF. The text is then parsed into rows and cells with a parser that respects
double-quoted fields and escaped quotes.
Each cell is optionally trimmed of leading and trailing whitespace, and runs of two or more spaces or tabs can be collapsed to a single space. Fully empty rows are dropped when that option is enabled. Finally each field is re-serialised and quoted only when it contains a comma, double quote, or newline, following RFC 4180 minimal-quoting rules.
Common problems this fixes
| Problem | Symptom | Cleaning option |
|---|---|---|
| UTF-8 BOM | First header reads id not id; column matching breaks | BOM removal |
| Windows CRLF line endings | Extra blank rows in some parsers; \r visible at line end | CRLF normalisation (always on in output) |
| Space-padded cells | " London " does not match "London" in a JOIN | Cell trimming |
| Double spaces in text | Inconsistent display and mismatched string comparisons | Collapse multiple spaces |
| Empty rows from spreadsheet export | Extra blank rows break row count assumptions | Drop empty rows |
When to leave options off
Cell trimming is safe for most numeric and text data, but leave it off if leading or trailing spaces carry meaning. Fixed-format reference codes (for example " A1" where the space is structural), right-padded legacy identifiers, and whitespace-significant markup can all be broken by trimming.
Similarly, number inference is not part of this tool — it cleans formatting only, so numeric strings like "1,000" (with a comma thousand separator) remain as strings. Strip the comma in your downstream tool or preprocessing step.
Typical workflow
- Export your CSV from the source system (database, Excel, Google Sheets).
- Paste it here and enable the options that match the known issues (BOM and CRLF are almost always safe to enable).
- Read the summary — confirm the counts match what you expected. If BOM count is 0 but you were certain the file had one, the source was already clean.
- Download the cleaned file and import it into the target system.
Running the cleaner before a database import or a pandas read_csv call prevents the most common “column not found” and silent mismatches that only show up later in the data pipeline. All processing runs locally — nothing is uploaded.