⚙️ Python subprocess Done Right: shlex, Timeouts, and check

Almost every Python automation script I have ever written eventually shells out to something: kubectl, terraform, git, a vendor CLI with no API. And almost every subprocess bug I have ever debugged came down to the same three mistakes: using shell=True when you did not need to, forgetting a timeout so the script hangs forever, and ignoring the exit code so a failure sails through silently.

This is a tour of how I use subprocess defensively, the patterns I trust, and why this is exactly the kind of code I let an AI draft but always review line by line.

Start with subprocess.run, not os.system

os.system and os.popen are relics. They run everything through a shell, give you almost no control over output, and make injection trivial. subprocess.run is the modern, safe default:

import subprocess

result = subprocess.run(
    ["git", "rev-parse", "HEAD"],
    capture_output=True,
    text=True,
    check=True,
)
commit = result.stdout.strip()

Three things matter here. The command is a list, not a string. text=True gives you str instead of bytes. And check=True raises CalledProcessError if the command exits non-zero so failures cannot be ignored.

Pass a list, avoid shell=True

The most common security bug in shell-outs is string interpolation into shell=True:

# DANGEROUS: filename could contain `; rm -rf ~`
subprocess.run(f"tar czf backup.tgz {filename}", shell=True)

If filename comes from anywhere a user or external system can influence, that is remote code execution. Pass arguments as a list and skip the shell entirely:

# SAFE: arguments are passed directly to exec, no shell parsing
subprocess.run(["tar", "czf", "backup.tgz", filename], check=True)

When the list form is used, there is no shell to interpret ;, |, $(), or globs. The argument is handed to the program verbatim.

Pro Tip: If you genuinely need shell features like pipes, prefer to do the piping in Python with multiple subprocess calls, or build the argument list with shlex.split() on a trusted, hard-coded template, never on user input.

shlex for the times you must build commands

Sometimes you are handed a command as a string (from a config file, say) and need to split it into arguments correctly. Naive .split() breaks on quoted paths. shlex.split handles quoting the way a shell would:

import shlex

cmd = 'kubectl get pods -n "my namespace" -o json'
args = shlex.split(cmd)
# ['kubectl', 'get', 'pods', '-n', 'my namespace', '-o', 'json']
subprocess.run(args, check=True)

And when you need to log a command safely or build one for display, shlex.quote escapes a single argument so it is shell-safe:

print("Running:", " ".join(shlex.quote(a) for a in args))

Always set a timeout

A subprocess with no timeout is a hang waiting to happen. A flaky network call, a CLI prompting for input on stdin, a hung remote host — any of these will freeze your automation indefinitely. Set a timeout on every call:

try:
    result = subprocess.run(
        ["terraform", "plan", "-no-color"],
        capture_output=True,
        text=True,
        timeout=300,
        check=True,
    )
except subprocess.TimeoutExpired:
    raise SystemExit("terraform plan timed out after 5 minutes")

When the timeout fires, the child is killed and TimeoutExpired is raised. I treat a missing timeout as a code-review blocker.

Handle the errors you can predict

check=True turns failures into exceptions, which is what you want most of the time. But catch them where you can do something useful, and surface the captured output so the error is debuggable:

try:
    subprocess.run(
        ["kubectl", "apply", "-f", "deploy.yaml"],
        capture_output=True,
        text=True,
        check=True,
        timeout=60,
    )
except subprocess.CalledProcessError as e:
    raise SystemExit(
        f"kubectl apply failed (exit {e.returncode}):\n{e.stderr}"
    )

Without capturing and re-raising e.stderr, your error log just says “Command failed” and you are left guessing.

Streaming output for long-running commands

capture_output=True buffers everything until the process exits, which is fine for short commands but useless when you want live output from a 20-minute terraform apply. For that, use Popen and read line by line:

import subprocess

with subprocess.Popen(
    ["terraform", "apply", "-auto-approve"],
    stdout=subprocess.PIPE,
    stderr=subprocess.STDOUT,
    text=True,
    bufsize=1,
) as proc:
    for line in proc.stdout:
        print(line, end="")
rc = proc.wait()
if rc != 0:
    raise SystemExit(f"terraform apply failed with code {rc}")

Merging stderr into stdout keeps the output ordered the way a human would see it in a terminal.

Where AI helps, and where I slow down

Translating a pile of shell commands into clean subprocess calls is tedious, mechanical work — exactly what an AI assistant is good at. I will paste a Bash script into ChatGPT or GitHub Copilot and ask it to convert each command into a list-form subprocess.run with check=True and a timeout. It is fast and usually correct.

But I treat the assistant as a fast junior engineer whose output I always review, because subprocess code touches the real system. Before anything runs against production I check:

• No stray shell=True crept in, especially around user-influenced strings.
• Every call has a timeout.
• Exit codes are checked, and stderr is surfaced on failure.
• No real credentials were pasted into the prompt for context. The AI never needs your secrets, and a generated script should read them from the environment at runtime, not have them baked in.

I keep that review checklist in the prompt workspace and route the riskier conversions through the code review dashboard before they go anywhere near a cluster.

Passing environment and input safely

Two more things ops scripts commonly need: a controlled environment and data piped to stdin. Pass env explicitly rather than mutating the global one, and use input= instead of writing to stdin by hand:

import os, subprocess

# A minimal, explicit environment for the child
child_env = {
    "PATH": os.environ["PATH"],
    "AWS_PROFILE": "deploy",
    "TF_IN_AUTOMATION": "1",
}
subprocess.run(
    ["terraform", "plan"],
    env=child_env,
    check=True,
    timeout=300,
)

# Feed data to stdin without a manual pipe
subprocess.run(
    ["kubectl", "apply", "-f", "-"],
    input=manifest_yaml,
    text=True,
    check=True,
    timeout=60,
)

Passing an explicit env is also a security control: the child only sees the variables you choose, so a stray secret in your shell environment does not silently leak into a subprocess (and possibly into its logs). I build the child environment from the specific keys a command needs rather than inheriting everything.

The input= form is the clean way to do echo ... | kubectl apply -f -. It hands the string to the child’s stdin and closes it, with no shell pipe and no Popen plumbing.

A small wrapper to enforce the defaults

Because forgetting a timeout is so easy, I often wrap subprocess.run once per project so the safe defaults are automatic:

import subprocess

def run(args, *, timeout=120, **kw):
    return subprocess.run(
        args,
        capture_output=True,
        text=True,
        check=True,
        timeout=timeout,
        **kw,
    )

Now every call in the codebase is captured, decoded, checked, and time-bounded by default. You only override when you have a reason.

Conclusion

Safe subprocess use is a short list: pass arguments as a list, never shell=True on untrusted input, always set a timeout, always check the exit code, and surface captured output on failure. Let an AI draft the conversions, then read them with a security mindset before they touch real infrastructure. Get those habits into a shared wrapper and the whole codebase inherits them for free.

More in the Bash and Python automation category. Grab reusable starters from the prompt library or a full set from the prompt packs.