♻️ Using AI to Make an Ansible Playbook Truly Idempotent

The first time I trusted a playbook in production, it bit me. It ran clean the first time, then on the second run it restarted a service that didn’t need restarting and took down a node mid-deploy. The playbook looked idempotent. It wasn’t. That gap — between “looks fine” and “reports no change when nothing changed” — is where most Ansible pain lives, and it’s exactly the kind of tedious audit work AI is good at accelerating.

I treat AI here like a fast junior engineer: it can read every task in a 600-line role faster than I can and flag the suspicious ones, but I review every flag and I never merge a “fix” without proving it in check-mode first.

Why idempotency is the whole point

A truly idempotent playbook reports changed=0 on the second run when nothing in the world has changed. That property is what lets you run Ansible on a schedule, in a pipeline, or in a panic at 2am without wondering whether you’re about to make things worse. Break it, and ansible-playbook becomes a loaded gun every time it executes.

The trouble is that Ansible doesn’t enforce idempotency. Most modules give it to you for free, but the moment you reach for command, shell, or a poorly-configured lineinfile, you’re on your own.

Hand the playbook to AI as a reviewer, not an author

I don’t ask AI to write the playbook. I ask it to audit one I already have. The prompt that works for me is specific:

“Review this Ansible role for idempotency violations. For each task, tell me whether a second run will report changed. Flag every command/shell task missing creates, removes, or changed_when. Don’t rewrite anything yet — just list the risks.”

That framing matters. A request to “fix this” gets you a confident rewrite you can’t trust. A request to “list the risks” gets you a checklist you can verify. Here’s the classic offender it catches every time:

- name: "Generate the app config"
  ansible.builtin.command: /opt/app/bin/render-config --out /etc/app/config.yaml

That runs on every single play, reports changed every time, and may restart things downstream. The AI-suggested fix, which I then review:

- name: "Generate the app config"
  ansible.builtin.command: /opt/app/bin/render-config --out /etc/app/config.yaml
  args:
    creates: /etc/app/config.yaml
  register: render_result
  changed_when: "'wrote' in render_result.stdout"

The creates guard skips the task if the file already exists; changed_when makes the change report honest when it does run. Good, but creates is too blunt if the config can legitimately change — so I push back and we land on a command that diffs first, or better, a real templating module.

Prefer modules over shell, every time

The single biggest idempotency win is replacing shell: glue with the right module. AI is genuinely fast at this translation because it has the module reference memorized. I’ll paste a crusty task:

- name: "Add the deploy user to docker group"
  ansible.builtin.shell: usermod -aG docker deploy

And get back the idempotent version, which only changes when the group membership actually differs:

- name: "Add the deploy user to docker group"
  ansible.builtin.user:
    name: deploy
    groups: docker
    append: true

Pro Tip: When AI suggests a module swap, ask it to name the module’s idempotency behavior explicitly — “does ansible.builtin.user with append: true report changed if the user is already in the group?” If it can’t justify the no-change path, the suggestion isn’t done.

Check-mode is non-negotiable

This is the rule I never break: every AI-suggested change runs in check-mode before it touches a real host.

ansible-playbook site.yml --check --diff --limit staging-01

The --diff flag shows me exactly what would change, line by line. The real proof of idempotency is two consecutive runs: apply once for real, then run --check again and confirm it reports zero changes.

ansible-playbook site.yml --limit staging-01
ansible-playbook site.yml --check --diff --limit staging-01   # must show changed=0

If that second command shows any changed, the task is lying about its state and AI’s “fix” failed. Note that not every module supports check-mode cleanly — command and shell get skipped unless you set check_mode: false deliberately, which is itself a signal that task needs a changed_when.

Watch for the sneaky non-idempotent patterns

A few patterns AI flags well once you tell it to look:

• lineinfile without an anchor. Without a precise regexp, repeated runs can append duplicate lines. Ask AI to verify the regexp matches the line it inserts.
• template into a file something else also writes. Two writers means perpetual churn. AI can spot the second writer if you give it the whole role.
• set_fact with timestamps or random values. These guarantee a diff every run. AI is good at catching lookup('pipe', 'date') buried in a default.
• get_url without checksum. Re-downloads can report changed even when the file is identical.

Keep AI away from the parts that matter most

There’s a hard line I hold. AI never sees my real group_vars, my vault password, or anything decrypted from ansible-vault. When I want it to reason about a task that references a secret, I substitute a placeholder:

- name: "Configure DB connection"
  ansible.builtin.template:
    src: db.conf.j2
    dest: /etc/app/db.conf
  vars:
    db_password: "REDACTED_PLACEHOLDER"

The structure is all the AI needs to reason about idempotency. The secret is mine.

A repeatable workflow

My loop looks like this: paste the role to AI for an idempotency audit, review the flagged tasks myself, accept the module swaps that I can justify, run check-mode against staging, run the double-run proof, then put the diff in front of a human teammate before it merges. If you want a place to keep prompts like the audit one above sharp, the prompt workspace and the broader prompt library are where I version mine.

The payoff is real: playbooks I can schedule without flinching. But the AI didn’t make them idempotent — check-mode and a human reviewer did. AI just made the audit fast. For more on where AI fits in IaC generally, the Infrastructure as Code category collects the rest of this series, and Claude remains my go-to for this kind of structured YAML review.

Idempotency is a discipline, not a feature. Let AI speed up the grunt work of finding the violations, then prove every fix yourself.