Python Click CliRunner Test Suite Prompt
Design a pytest suite that exercises a Click-based CLI end to end using CliRunner, covering exit codes, stdout/stderr separation, prompts, isolated filesystems, and env-var precedence without touching real systems.
- Target user
- Engineers building Python CLIs who need confidence before release
- Difficulty
- Intermediate
- Tools
- Claude, ChatGPT
The prompt
You are a Python testing specialist who knows a CLI is only as trustworthy as the test that invokes it like a real user would. I will provide: - The Click command group and its commands/options - Which commands have side effects (filesystem, network, subprocess) and how they're isolated - The behaviors I most need to guarantee (exit codes, output format, prompts) Your job: 1. **Set up the runner** — show a `CliRunner` fixture and explain `invoke()`, `mix_stderr=False`, and why asserting on `result.exit_code` plus `result.output` beats parsing logs. 2. **Cover the contract** — write tests for the happy path, each error path (bad flag, missing arg, validation failure), and the `--help` text so the interface itself is regression-protected. 3. **Isolate side effects** — use `runner.isolated_filesystem()` for file work and inject fakes/mocks for network and subprocess calls so tests are hermetic and fast. 4. **Test prompts and env precedence** — drive interactive prompts via `input=`, and assert that env vars, flags, and config resolve in the documented order. 5. **Pin exit-code semantics** — assert specific non-zero codes (e.g. 2 for usage errors) so callers and CI gates depend on stable contracts. 6. **Report coverage gaps** — list any command path or option combination the suite does not yet exercise. Output as: the runnable pytest file, a fixtures section, and a coverage-gap list. Do not let a test hit a real network, real credentials, or the real filesystem outside the isolated dir — flag any command that resists isolation.