Bash Batch File Conversion Pipeline Prompt
Create a Bash pipeline that converts a directory of files (images, audio, or docs) in bulk with parallelism, skipping already-converted outputs idempotently.
- Target user
- Engineers automating repetitive bulk media or document conversion
- Difficulty
- Intermediate
- Tools
- Claude, ChatGPT
The prompt
You are a senior automation engineer who builds robust batch processing pipelines that survive being re-run on partial output. I will provide: - The input directory and file glob - The conversion command and the output format/extension - The desired parallelism and output directory Your job: 1. **Scaffold strictly** — `#!/usr/bin/env bash`, `set -euo pipefail`, `IFS=$'\n\t'`, and preflight-check that the conversion binary exists with `command -v`. 2. **Handle tricky names** — iterate with `find -print0` and a null-delimited loop so spaces and newlines in filenames never break processing. 3. **Be idempotent** — skip any input whose output already exists and is newer than the source; re-running only processes missing or stale files. 4. **Convert atomically** — write each output to a temp path, then `mv` into place only on success, so a killed job never leaves a truncated file. 5. **Parallelize bounded** — run conversions concurrently with a capped worker count via `xargs -P` or GNU parallel. 6. **Aggregate errors** — continue past individual failures, collect them, and exit non-zero if any conversion failed. 7. **Dry-run** — `--dry-run` prints the planned conversions without running them. Output as: (a) the script, (b) an example invocation, (c) notes on tuning parallelism. Convert into temp files and atomically rename; an interrupted batch must leave only complete outputs.