🐍 Embedding Ansible in Python Apps With ansible-runner and AI

Sooner or later someone wants Ansible to run from inside an application instead of from a shell — a self-service portal that provisions environments, a remediation worker that responds to alerts, a webhook that kicks off a deploy. The wrong way to do this is to subprocess.Popen(['ansible-playbook', ...]) and parse stdout with regex. The right way is ansible-runner, the library and CLI that AWX itself uses to execute playbooks. It gives you a stable interface, structured events instead of scraped text, and isolated run directories so concurrent runs don’t trample each other.

I use AI to wire up the runner config and the event handling, because the API surface is broad and the docs are terse, but I keep a firm hand on the parts that matter: how secrets get in, how results come out, and how failures surface. An embedded automation engine that hides a failed run is worse than no automation at all.

What ansible-runner gives you

ansible-runner runs a playbook inside a defined directory structure (the “private data dir”) and emits a stream of events you can consume programmatically. Instead of guessing whether a run worked from exit codes and text, you get the run status, per-event data, and stats as real objects.

import ansible_runner

r = ansible_runner.run(
    private_data_dir='/tmp/run1',
    playbook='site.yml',
    inventory='inventories/prod',
)

print(r.status)   # 'successful' / 'failed'
print(r.rc)       # return code
print(r.stats)    # {'ok': {...}, 'changed': {...}, 'failures': {...}}

r.status and r.stats are the contract you build on. A worker that checks r.status == 'successful' and inspects r.stats['failures'] is reliable in a way that grepping stdout never is.

The private data dir structure

The runner expects a directory layout, and understanding it is half the battle. AI is genuinely useful here because the layout is easy to get slightly wrong:

/tmp/run1/
├── env/
│   ├── envvars        # environment variables for the run
│   └── extravars      # extra-vars (JSON/YAML), kept out of argv
├── inventory/
│   └── hosts
└── project/
    └── site.yml       # your playbook

The reason extravars lives in a file rather than on the command line matters: arguments passed via argv show up in process listings, which is a bad place for anything sensitive. Putting extra-vars in env/extravars keeps them out of ps output. This is the kind of detail I specifically ask AI to get right.

Show me how to lay out an ansible-runner private_data_dir to run site.yml against a prod inventory, passing app_version and a vaulted DB password as extra-vars via the env/extravars file (not argv), with the vault password supplied through env/passwords. Then show the Python to invoke it and check status and stats.

Streaming events instead of scraping output

The feature that makes ansible-runner worth adopting is the event callback. Instead of waiting for the run to finish and parsing a wall of text, you handle each event as it happens — perfect for streaming progress to a UI or logging structured records:

def handle_event(event):
    if event['event'] == 'runner_on_failed':
        host = event['event_data']['host']
        task = event['event_data']['task']
        # push a structured failure record somewhere useful
        log_failure(host, task)

r = ansible_runner.run(
    private_data_dir='/tmp/run1',
    playbook='site.yml',
    event_handler=handle_event,
)

Each event carries event_data with the host, task, and result. This is how AWX shows you live task output, and it’s how your app can react to a specific task failing instead of discovering after the fact that “something went wrong.”

Passing secrets without leaking them

Secrets are where embedded Ansible most often goes wrong, because it’s tempting to just stuff a password into the extra-vars dict and move on. Two rules keep this safe. First, sensitive extra-vars go in the env/extravars file, never on argv. Second, the vault password is supplied through the runner’s passwords mechanism, not hardcoded:

r = ansible_runner.run(
    private_data_dir='/tmp/run1',
    playbook='site.yml',
    extravars={'app_version': '1.4.2'},   # non-sensitive only here
    passwords={'^Vault password:\\s*$': vault_pass_from_secret_store},
)

The app_version is fine in extravars; the vault password comes from your secrets store and is matched against the prompt pattern. I make a point of asking AI to keep secrets out of argv and out of any log line, and then I verify it by checking the run artifacts and process listing during a test run. A worker that logs the full extra-vars dict at debug level will cheerfully write your DB password to disk.

Pro Tip: ansible-runner writes run artifacts (stdout, events, stats) into the private data dir. Make sure those directories are mode-restricted and cleaned up, because the stdout artifact can contain anything your playbook printed — including secrets a careless debug task exposed.

Isolation and concurrency

Because each run gets its own private data dir, you can run many playbooks concurrently without them stepping on each other — as long as you give each a unique directory. For stronger isolation, ansible-runner can execute inside a container using an execution environment image, which pins the exact Ansible version, collections, and dependencies so a run on your worker behaves identically to a run anywhere else:

r = ansible_runner.run(
    private_data_dir='/tmp/run1',
    playbook='site.yml',
    process_isolation=True,
    container_image='our-registry/ee-base:2025.10',
)

That container approach is how you stop the classic “works on my worker, fails on the new one” drift, because the execution environment image is the dependency contract.

Making failures impossible to ignore

The single most important thing about embedding Ansible is that failures must surface. I always wire the worker to treat r.status != 'successful' as a hard failure that propagates to whatever scheduled the run — a non-zero job result, an alert, a visible error in the portal. The trap is a fire-and-forget worker that runs a playbook, ignores the status, and reports success while the deploy quietly failed. The structured status and stats exist precisely so you never have to guess; use them.

ansible-runner turns Ansible into a proper library you can build a product on, and AI is a fine pair for wiring up the config and event handling. Just keep ownership of the three things that bite: secrets stay off argv and out of logs, runs stay isolated, and failures always surface.

For the dependency-pinning side of this, see Ansible execution environments and collections done right and the AWX self-service guide. The full AI for Ansible category and the Ansible prompts have more.