Python Structured Logging for Automation Scripts Prompt
Add production-grade structured logging to a Python automation script — JSON logs, correlation IDs, levels, redaction of secrets, and rotation — so cron and systemd runs are debuggable after the fact.
- Target user
- Engineers making batch scripts observable in shared log pipelines
- Difficulty
- Intermediate
- Tools
- Claude, ChatGPT
The prompt
You are a senior engineer who has spent too many on-call shifts grepping `print()` output that lost the one detail you needed. Add structured, production-grade logging to a Python automation script. I will provide: - The script and how it currently logs (prints, ad-hoc logging, nothing) - Where logs land (stdout for systemd/journald, a file, a shipper like Loki/ELK) - What sensitive values flow through it (tokens, PII, connection strings) - Whether multiple instances or threads run concurrently Your job: 1. **Choose the setup** — configure the stdlib `logging` module via `dictConfig`; emit JSON when shipped to an aggregator and human-readable lines for interactive runs, selected by a `--log-format` flag or TTY detection. Recommend `structlog` only if the gains justify the dependency, and explain the tradeoff. 2. **Make every line queryable** — include timestamp (ISO 8601 UTC), level, logger name, message, and structured fields (not interpolated into the message string). For systemd, log to stdout and let journald capture it rather than managing files. 3. **Correlation IDs** — generate a run ID at startup and attach it (plus relevant entity IDs) to every record via a `LoggerAdapter` or `contextvars`, so all lines from one run — and one work item — are trivially filterable. 4. **Levels with intent** — DEBUG for wire details, INFO for lifecycle milestones, WARNING for handled-but-notable, ERROR for failed work items, CRITICAL for aborting. Show how `--verbose`/`-v` flags map to levels. 5. **Redact secrets** — install a logging filter that masks known sensitive keys and pattern-matches tokens/passwords before they hit any handler, so a careless `logger.info(config)` cannot leak credentials. This is non-negotiable. 6. **Rotation (file path)** — if writing files, use `RotatingFileHandler`/`TimedRotatingFileHandler` with size/age caps, or hand off to `logrotate`; never let an automation log grow unbounded on a small disk. 7. **Exceptions** — log tracebacks with `logger.exception` inside handlers, attach the run/item context, and ensure a top-level handler logs a structured fatal record with a non-zero exit. 8. **Bash interop** — for mixed pipelines, show a matching JSON log helper in Bash so shell and Python lines share fields and the run ID propagates via env var. Output: (a) the `dictConfig` and JSON formatter, (b) the redaction filter with tests proving secrets never appear, (c) the correlation-ID adapter, (d) before/after of one noisy script, (e) a query cheatsheet for the chosen aggregator. Be opinionated: structured fields over string interpolation, stdout for systemd, and redaction enabled by default.
Run this prompt with AI
Test it, get an AI-improved version, or compare models — live in the Prompt Workspace. No copy-paste.
Related prompts
-
Bash Leveled Logging Library Prompt
Build a small, sourceable Bash logging library with DEBUG/INFO/WARN/ERROR levels, timestamps, TTY-aware color, and a LOG_LEVEL threshold — so your scripts emit consistent, greppable output to stderr.
-
Python Multi-Log Merge and Timeline Correlator Prompt
Build a Python tool that merges several log files with different timestamp formats into one time-ordered timeline, tags each line with its source, and correlates events across services for incident analysis
-
Python logging.config dictConfig Setup Prompt
Configure Python's logging via a single declarative dictConfig dictionary with multiple handlers, formatters, and per-module log levels instead of scattered basicConfig calls
-
Bash Script Machine-Readable JSON Output Prompt
Retrofit a bash script with a clean --json output mode using jq, separating human stdout from machine output, emitting valid structured results and proper exit codes so the script composes into pipelines and CI.
More Bash & Python Automation prompts & error guides
Browse every Bash & Python Automation prompt and troubleshooting guide in one place.
Reading prompts? Get all 500 in one free PDF
500 battle-tested, copy-paste AI prompts engineered by a senior systems engineer — every one with fill-in placeholders and safety/back-out notes. Drop your email and it's yours.
- 500 prompts: Linux · Kubernetes · Terraform · OpenStack · GitLab · Docker · Monitoring · Incident Response
- Instant PDF download — yours free, forever
- Plus one practical AI-workflow email a week (no spam)
Single opt-in · unsubscribe anytime · no spam.