🔒 Automating systemd Unit Hardening with AI

Most services running on a Linux box have far more privilege than they need. A web app that only reads two directories and listens on one port is, by default, allowed to write anywhere root can, load kernel modules, and call reboot(). systemd ships a whole arsenal of sandboxing directives to fix that — but almost nobody uses them, because writing them by hand is tedious and easy to get wrong.

I’ve spent the last few years bolting these directives onto units, and the workflow that finally stuck is: let an AI draft the hardening block from the unit’s actual behavior, then I review every line before it touches a running service. The model is genuinely good at remembering the full menu of Protect* and Restrict* options I’d otherwise have to look up. It’s also confidently wrong often enough that I’d never let it apply anything unattended.

Start with the security score

Before touching anything, get a baseline. systemd-analyze security rates every directive a unit could set and gives an overall exposure score from 0 (locked down) to 10 (wide open):

systemd-analyze security nginx.service

You’ll get a long table with verdicts like ✗ ProtectSystem= is not enabled and an exposure number. Most stock units land around 8.0–9.5 — effectively unsandboxed. That table is the perfect input for an AI prompt, because it’s a checklist of exactly what’s missing.

Pipe the report and the unit file into your assistant and ask for an override. I keep a saved version of this prompt in my linux admin prompts collection so I don’t retype it:

Here is a systemd unit and its systemd-analyze security output. Draft a drop-in override at /etc/systemd/system/<svc>.d/hardening.conf that raises the security score. The service reads from /var/www and /etc/nginx, writes logs to /var/log/nginx, and binds ports 80 and 443. Explain each directive and flag any that might break it.

The directives that matter most

You don’t need all forty. A handful does most of the work:

[Service]
ProtectSystem=strict
ProtectHome=true
PrivateTmp=true
NoNewPrivileges=true
ProtectKernelTunables=true
ProtectKernelModules=true
ProtectControlGroups=true
RestrictAddressFamilies=AF_INET AF_INET6
RestrictNamespaces=true
MemoryDenyWriteExecute=true
SystemCallFilter=@system-service
SystemCallArchitectures=native

ProtectSystem=strict mounts the entire filesystem read-only except a few allowed paths. Pair it with ReadWritePaths=/var/log/nginx for the directories the service legitimately writes. NoNewPrivileges=true is the single highest-value line — it stops the process and all its children from ever gaining privileges through setuid binaries.

Pro Tip: MemoryDenyWriteExecute=true breaks anything with a JIT — Node’s V8, the JVM, some Python extensions. If the service won’t start after you add it, that’s the first directive to drop. AI frequently suggests it for runtimes where it can’t work, so treat it as the prime suspect.

Drafting the override, not editing the vendor unit

Never edit the file in /lib/systemd/system/ — a package update will clobber it. Use a drop-in:

sudo systemctl edit nginx.service

This opens an override at /etc/systemd/system/nginx.service.d/override.conf that layers on top of the vendor unit. Paste in the AI’s draft, save, then:

sudo systemctl daemon-reload
sudo systemctl restart nginx.service
systemctl status nginx.service

If you’d rather review the full picture, our code review helper is handy for eyeballing a generated override before you commit it, and the prompt workspace lets you iterate on the hardening prompt without leaving the browser.

Test in the harness, not in production

Here’s the rule I never break: the model drafts, a human applies. systemd gives you a safe way to dry-run a single directive set:

systemd-run --pty \
  --property=ProtectSystem=strict \
  --property=PrivateTmp=true \
  --property=NoNewPrivileges=true \
  /usr/bin/your-binary --check

This spins up a transient unit with exactly those sandboxing options and runs your binary inside it. If it fails, you’ve learned which directive is too tight without touching the real service. AI is a fast junior engineer here — great at producing the candidate directive list, useless at knowing whether your specific binary will tolerate it. Only the test tells you that.

Reading failures back into the next draft

When a hardened service breaks, the journal usually says exactly which sandbox call was denied:

journalctl -u nginx.service -b --no-pager | tail -40

Look for lines mentioning Operation not permitted, EPERM, or a denied syscall. Feed that back to the model: “The service now fails with this error after I added SystemCallFilter=@system-service. Which syscall group do I need to allow?” It’ll usually suggest adding @network-io or ~@privileged and explain the reasoning. You verify against the actual restart — never just trust the explanation.

For services you can’t afford to break, I stage the override on an identical test box first. The whole point of keeping a human in the loop is catching the difference between “this looks right” and “this works on this exact kernel and runtime version.”

Build a reusable hardening profile

Once you’ve hardened nginx, the next web service is 80% the same. Save your verified overrides as a template. I keep a small library of these — one for stateless HTTP services, one for things that need raw sockets, one for daemons that legitimately write across the filesystem. The prompt packs and the broader prompts library are where I stash the prompt scaffolding that generates a first draft from a unit file plus its security score.

The payoff compounds. After the third or fourth service, you’re not asking the AI to invent anything — you’re asking it to adapt a known-good profile to a new unit, which is exactly the pattern-matching it’s reliable at.

Never hand the model your servers

One hard line: the AI drafts text, it does not run commands on your hosts. Don’t paste production credentials, don’t wire it into a tool that has systemctl rights, don’t let it daemon-reload on its own. A hardening directive that’s subtly wrong can lock a service out of a path it needs at 3 a.m., and you want a human who understands the blast radius standing between the draft and the running system.

Conclusion

systemd sandboxing is one of the highest-leverage, lowest-effort security wins available on a Linux server, and the only reason it stays unused is the tedium of writing the directives. AI removes the tedium: it remembers the full menu, drafts a sensible override from the security report, and explains each line. Your job is the part that actually requires judgment — testing the draft in a transient unit, reading the failures, and deciding what’s safe to ship. Draft fast, apply slowly, and keep systemd-analyze security honest about how far you’ve come.