Linux Error Guide: 'Failed to start' systemd Service Won't Start
Fix the systemd 'Failed to start unit' error: diagnose status=203/EXEC missing binaries, bad WorkingDirectory/User, failed dependencies, start-limit-hit, and sandboxing.
- #linux
- #troubleshooting
- #errors
- #systemd
Overview
A systemd “Failed to start” error means the service manager attempted to run a unit’s start commands and the control process exited non-zero, was never spawned, or timed out before signalling readiness. systemd marks the unit failed, refuses to consider it active, and (depending on the Restart= policy) may stop trying altogether.
You will see this on the command line after systemctl start:
Job for myapp.service failed because the control process exited with error code.
See "systemctl status myapp.service" and "journalctl -xeu myapp.service" for details.
The headline message is deliberately vague — the real cause is in the unit’s exit status, which systemd records as a status= code (for example 203/EXEC or 217/USER). It occurs whenever the unit is started: at boot, on a manual systemctl start, or when another unit pulls it in as a dependency. The same unit can start cleanly on one host and fail on another because the binary path, user, or SELinux context differs.
Symptoms
systemctl start myapp.servicereturns the “control process exited with error code” message and a non-zero exit.systemctl statusshowsActive: failed (Result: exit-code)orResult: start-limit-hit.- The unit flaps: it starts, dies, restarts a few times, then stops with “start request repeated too quickly”.
- The journal records
code=exited, status=203/EXEC(or200/CHDIR,217/USER).
systemctl status myapp.service --no-pager
× myapp.service - My Application Daemon
Loaded: loaded (/etc/systemd/system/myapp.service; enabled; preset: enabled)
Active: failed (Result: exit-code) since Tue 2026-06-23 14:02:11 UTC; 6s ago
Process: 4821 ExecStart=/usr/local/bin/myapp --config /etc/myapp/myapp.conf (code=exited, status=203/EXEC)
Main PID: 4821 (code=exited, status=203/EXEC)
CPU: 2ms
Jun 23 14:02:11 host01 systemd[1]: myapp.service: Main process exited, code=exited, status=203/EXEC
Jun 23 14:02:11 host01 systemd[1]: myapp.service: Failed with result 'exit-code'.
Jun 23 14:02:11 host01 systemd[1]: Failed to start myapp.service - My Application Daemon.
Common Root Causes
1. ExecStart binary missing or wrong path (status=203/EXEC)
203/EXEC means systemd found the unit but could not execute the program named in ExecStart= — the path is wrong, the file is absent, or it is not marked executable.
systemctl show myapp.service -p ExecStart --no-pager
ls -l /usr/local/bin/myapp
ExecStart={ path=/usr/local/bin/myapp ; argv[]=/usr/local/bin/myapp --config /etc/myapp/myapp.conf ; ... }
ls: cannot access '/usr/local/bin/myapp': No such file or directory
The binary is not where the unit expects it. A -rw-r--r-- (no execute bit) file produces the same 203/EXEC.
2. Bad WorkingDirectory or User (status=200/CHDIR, 217/USER)
200/CHDIR means the WorkingDirectory= does not exist or is unreadable; 217/USER means the User=/Group= cannot be resolved on this host.
systemctl show myapp.service -p WorkingDirectory -p User -p Group --no-pager
id myapp 2>&1
ls -ld /var/lib/myapp 2>&1
WorkingDirectory=/var/lib/myapp
User=myapp
Group=myapp
id: 'myapp': no such user
ls: cannot access '/var/lib/myapp': No such file or directory
The myapp account was never created and the working directory is missing, so the process is killed before ExecStart ever runs.
3. A unit dependency failed or ordering is wrong
If a unit listed in Requires= fails, systemd will not start the dependent unit. A missing After= can also let the service start before a resource (network, mount, database socket) is ready.
systemctl list-dependencies myapp.service --no-pager
systemctl status postgresql.service --no-pager | head -3
myapp.service
● ├─postgresql.service
● ├─network-online.target
× └─myapp-data.mount
× postgresql.service - PostgreSQL database server
Active: failed (Result: exit-code) since Tue 2026-06-23 14:01:48 UTC; 30s ago
postgresql.service (a Requires=) is failed and the myapp-data.mount did not mount, so myapp.service is blocked from starting.
4. The application itself exits non-zero (port in use, missing config)
If ExecStart runs but the app exits non-zero — a config file it cannot read, or a TCP port already bound — systemd reports a plain status=1 (or the app’s own exit code) and “Failed to start”.
journalctl -xeu myapp.service --no-pager | tail -6
ss -ltnp | grep ':8080'
Jun 23 14:05:02 host01 myapp[5120]: FATAL: listen tcp 0.0.0.0:8080: bind: address already in use
Jun 23 14:05:02 host01 systemd[1]: myapp.service: Main process exited, code=exited, status=1/FAILURE
Jun 23 14:05:02 host01 systemd[1]: Failed to start myapp.service - My Application Daemon.
LISTEN 0 4096 0.0.0.0:8080 0.0.0.0:* users:(("nginx",pid=1334,fd=8))
Port 8080 is already held by nginx, so the app aborts at startup.
5. Start-limit-hit (“start request repeated too quickly”)
With Restart=always, a service that crashes immediately will be restarted until it trips StartLimitBurst within StartLimitIntervalSec. systemd then refuses further restarts and reports Result: start-limit-hit.
systemctl status myapp.service --no-pager | grep -E 'Active|Result'
journalctl -u myapp.service --no-pager | grep -i 'start request repeated' | tail -2
Active: failed (Result: start-limit-hit) since Tue 2026-06-23 14:08:40 UTC; 12s ago
Jun 23 14:08:40 host01 systemd[1]: myapp.service: Start request repeated too quickly.
Jun 23 14:08:40 host01 systemd[1]: myapp.service: Failed with result 'start-limit-hit'.
The underlying crash must be fixed; clear the counter with systemctl reset-failed myapp.service before retrying.
6. Bad unit syntax, not reloaded, or sandboxing blocks writes
An edited unit file that was never reloaded, a syntax error, or a hardening directive (ProtectSystem=strict, ReadOnlyPaths=, ProtectHome=) that makes a needed path read-only all produce a start failure. Type=notify services that never call sd_notify(READY=1) time out the same way.
systemd-analyze verify /etc/systemd/system/myapp.service
systemctl show myapp.service -p ProtectSystem -p ReadWritePaths -p Type --no-pager
/etc/systemd/system/myapp.service:12: Unknown key name 'ExecStartt' in section 'Service', ignoring.
ProtectSystem=strict
ReadWritePaths=
Type=notify
A typo’d ExecStartt is ignored (so nothing runs), and ProtectSystem=strict with no ReadWritePaths= will block the app from writing its logs or PID file. After any edit, run systemctl daemon-reload.
Diagnostic Workflow
Step 1: Read the status and the exit code
systemctl status myapp.service --no-pager
Note the Active: result and the status= number on the Process: line — 203/EXEC, 200/CHDIR, 217/USER, start-limit-hit, or a plain app exit code each point at a different cause.
Step 2: Read the full journal for this unit
journalctl -xeu myapp.service --no-pager | tail -40
The -x adds catalog hints and -e jumps to the end. Application FATAL lines, “address already in use”, or “Permission denied” appear here.
Step 3: Verify the unit file and that it was reloaded
systemd-analyze verify /etc/systemd/system/myapp.service
systemctl cat myapp.service
sudo systemctl daemon-reload
systemd-analyze verify flags unknown keys and bad references; systemctl cat shows the effective unit including any drop-ins.
Step 4: Validate the execution context
systemctl show myapp.service \
-p ExecStart -p WorkingDirectory -p User -p Group -p ProtectSystem -p ReadWritePaths --no-pager
id "$(systemctl show -p User --value myapp.service)" 2>&1
ls -l "$(systemctl show -p ExecStart --value myapp.service | awk '{print $1}')" 2>&1
Confirm the binary exists and is executable, the user resolves, and the working/write directories exist.
Step 5: Fix the cause, reset state, and retry
sudo systemctl daemon-reload
sudo systemctl reset-failed myapp.service
sudo systemctl start myapp.service
systemctl is-active myapp.service
reset-failed clears a start-limit-hit counter so the restart logic runs again.
Example Root Cause Analysis
A newly deployed metrics-agent.service goes straight to failed after systemctl start, with the generic “control process exited with error code” message.
systemctl status shows the exit code:
Process: 6044 ExecStart=/opt/metrics-agent/bin/agent --config /etc/metrics/agent.yml (code=exited, status=203/EXEC)
Active: failed (Result: exit-code)
203/EXEC says systemd could not execute the binary. Checking the path:
systemctl show metrics-agent.service -p ExecStart --value
ls -l /opt/metrics-agent/bin/agent
{ path=/opt/metrics-agent/bin/agent ; argv[]=/opt/metrics-agent/bin/agent --config /etc/metrics/agent.yml ; ... }
-rw-r--r-- 1 root root 18472960 Jun 23 13:50 /opt/metrics-agent/bin/agent
The binary is present but has mode -rw-r--r-- — the execute bit was lost when it was copied off a Windows-formatted artifact tarball. systemd cannot exec a non-executable file, hence 203/EXEC.
Fix: restore the execute bit and restart:
sudo chmod 0755 /opt/metrics-agent/bin/agent
sudo systemctl reset-failed metrics-agent.service
sudo systemctl start metrics-agent.service
systemctl is-active metrics-agent.service
The service execs cleanly and reports active.
Prevention Best Practices
- Run
systemd-analyze verifyon every unit file in CI before it ships — it catches unknown keys and brokenExecStart/After=references that otherwise only fail at boot. - Always
systemctl daemon-reloadafter editing a unit; bake it into your config-management handler so a stale unit never runs. - Create the
User=,Group=, andWorkingDirectory=with the package or your provisioning tool, not by hand, so217/USERand200/CHDIRcannot happen on a fresh host. - When you harden a unit with
ProtectSystem=strictorReadOnlyPaths=, declare every writable path inReadWritePaths=and test the start, not just the syntax. - Set realistic
StartLimitIntervalSec/StartLimitBurstand alert onResult: start-limit-hitso a crash-looping service is noticed before it silently stays down. More hardening tips live in the Linux admins guides. - For fast triage of an unfamiliar failure, the free incident assistant can turn a
journalctl -xeudump into the likely exit-code cause.
Quick Command Reference
# Status and the all-important exit code
systemctl status myapp.service --no-pager
# Full journal for the unit, with catalog hints
journalctl -xeu myapp.service --no-pager | tail -40
# Validate syntax and show the effective unit (with drop-ins)
systemd-analyze verify /etc/systemd/system/myapp.service
systemctl cat myapp.service
# Inspect the execution context
systemctl show myapp.service \
-p ExecStart -p WorkingDirectory -p User -p Group -p ProtectSystem -p ReadWritePaths -p Type --no-pager
# Check the binary and user resolve
ls -l /usr/local/bin/myapp
id myapp
# Is something already on the port?
ss -ltnp | grep ':8080'
# Reload after edits, clear the failure counter, retry
sudo systemctl daemon-reload
sudo systemctl reset-failed myapp.service
sudo systemctl start myapp.service
Conclusion
A systemd “Failed to start” error is only the headline — the status= exit code in systemctl status tells you which layer broke. Read that code, then the journalctl -xeu output, before changing anything. The usual root causes:
- The
ExecStartbinary is missing, mispathed, or not executable (203/EXEC). - A bad
WorkingDirectoryor unresolvableUser/Group(200/CHDIR,217/USER). - A failed
Requires=dependency or a missingAfter=ordering. - The application exits non-zero on its own — port already in use or unreadable config.
- A crash loop that trips
start-limit-hit(“start request repeated too quickly”). - Bad unit syntax, a unit never reloaded, or sandboxing/SELinux blocking a needed write.
Match the exit code to the cause, fix it, daemon-reload, reset-failed, and start again — the fix is almost always a path, a permission, or a context the unit assumes but the host does not have.
Download the Free 500-Prompt DevOps AI Toolkit
500 battle-tested, copy-paste AI prompts engineered by a senior systems engineer — every one with fill-in placeholders and safety/back-out notes. Drop your email and it's yours.
- 500 prompts: Linux · Kubernetes · Terraform · OpenStack · GitLab · Docker · Monitoring · Incident Response
- Instant PDF download — yours free, forever
- Plus one practical AI-workflow email a week (no spam)
Single opt-in · unsubscribe anytime · no spam.