Python CSV Data-Quality Validator Prompt
Build a Python CLI that validates and cleans CSV/tabular data against declarative quality rules — types, ranges, required fields, uniqueness, referential checks — and emits a structured report of every violation
- Target user
- Engineers building data-ingestion and ETL automation in Python
- Difficulty
- Intermediate
- Tools
- Claude, ChatGPT, Cursor
The prompt
You are a data-engineering-minded Python developer who refuses to let malformed CSV rows flow silently into a pipeline, and you build validators that surface every bad row before ingestion. I will provide: - A sample of the CSV (or its columns) and the quality rules per column - The rule set: required/non-null, expected type (int/float/date/enum), value ranges or allowed sets, regex patterns, uniqueness, and cross-field/referential constraints - What to do with bad rows: reject the file, quarantine bad rows, or coerce/clean where safe Your job: 1. **Declarative schema** — express the column rules as data (e.g. a dict or dataclass per column), not scattered `if` statements, so rules are readable and testable. 2. **Stream, don't slurp** — read with `csv.DictReader` (or pandas only if justified) so large files don't blow memory, and validate row by row while tracking line numbers for error reporting. 3. **Report precisely** — for every violation, record file, line number, column, the offending value, and which rule failed; aggregate into a summary (counts per rule) plus a detailed list, and emit both human-readable and JSON output. 4. **Clean only when safe** — offer opt-in normalization (trim whitespace, parse dates, canonicalize enums) that is explicit and logged; never silently mutate data the caller did not ask to clean. 5. **Separate good from bad** — write validated rows and quarantined bad rows to separate outputs so downstream stages only ever see clean data. 6. **Exit meaningfully** — non-zero exit when the failure rate exceeds a configurable threshold, so CI/pipelines can gate on data quality. 7. **Test** — pytest cases covering each rule type, edge cases (empty file, header-only, embedded newlines/quotes, encoding), and the threshold gate. Output as: (a) the schema/rule definition, (b) the validator CLI (argparse), (c) the report formatter, (d) the pytest suite. Bias toward: declarative rules, precise per-cell error reporting, streaming over loading everything into memory, and never silently altering data.
Run this prompt with AI
Test it, get an AI-improved version, or compare models — live in the Prompt Workspace. No copy-paste.
Related prompts
-
Python Regex Extraction and Validation Toolkit Prompt
Build correct, readable, and well-tested regular expressions in Python for parsing logs, validating input, and extracting structured fields — with named groups, verbose mode, and catastrophic-backtracking checks.
-
Bash .env File Loader and Validator Prompt
Write a safe Bash loader that parses a .env file, validates required and typed variables, and exports them — without the security and quoting footguns of blindly sourcing untrusted env files.
-
Bash Dependency Preflight Check Prompt
Generate a reusable preflight block that verifies required commands, versions, env vars, and permissions before a Bash script does any real work — failing fast with actionable messages instead of cryptic mid-run errors.
-
Python Pydantic Settings Config Prompt
Replace ad-hoc os.environ access and scattered config parsing with a typed pydantic-settings model — validated env vars, .env loading, nested settings, secrets, and clear startup errors instead of runtime KeyErrors.
More Bash & Python Automation prompts & error guides
Browse every Bash & Python Automation prompt and troubleshooting guide in one place.
Reading prompts? Get all 500 in one free PDF
500 battle-tested, copy-paste AI prompts engineered by a senior systems engineer — every one with fill-in placeholders and safety/back-out notes. Drop your email and it's yours.
- 500 prompts: Linux · Kubernetes · Terraform · OpenStack · GitLab · Docker · Monitoring · Incident Response
- Instant PDF download — yours free, forever
- Plus one practical AI-workflow email a week (no spam)
Single opt-in · unsubscribe anytime · no spam.