🚨 Linux Error Guide: 'Job timed out' systemd Start Timeouts and Hung Units

Exact Error Message

When a systemd unit fails to finish starting within its allotted window, you will see messages like these in the journal:

systemd[1]: myapp.service: start operation timed out. Terminating.
systemd[1]: myapp.service: Failed with result 'timeout'.
systemd[1]: Failed to start MyApp Service.
systemd[1]: Job myapp.service/start timed out.
systemd[1]: Timed out waiting for device /dev/disk/by-uuid/4b1f...c92a.
systemd[1]: dev-disk-by\x2duuid-4b1f...c92a.device: Job dev-disk-by\x2duuid-4b1f...c92a.device/start timed out.
systemd[1]: Dependency failed for /mnt/data.
systemd[1]: data.mount: Job data.mount/start failed with result 'dependency'.

You may also encounter the related boot-time phrasing:

systemd[1]: Job dev-disk-by\x2duuid-4b1f...c92a.device/start timed out.
[ TIME ] Timed out waiting for device /dev/disk/by-uuid/4b1f...c92a.
[DEPEND] Dependency failed for /mnt/data.

The common thread is the word timed out attached to a Job, and a final result of timeout or dependency.

What the Error Means

systemd starts every unit as a transient job. Each job has a maximum amount of time it is allowed to spend in the “starting” state. If the unit does not reach the “active/running” state before that timer expires, systemd considers the start a failure, sends SIGTERM (then SIGKILL), and records the result as timeout.

The size of that window depends on the unit type and configuration:

• Service units are governed by TimeoutStartSec=, which defaults to DefaultTimeoutStartSec= (90 seconds on most distributions).
• Device units wait up to DefaultTimeoutStartSec= (again, 90s) for the kernel/udev to announce the device. A missing or renamed disk never appears, so the job times out.
• Mount units generated from /etc/fstab inherit the device timeout plus their own mount timeout. A mount that depends on a missing device fails with dependency after the device job times out.

Crucially, how systemd decides a service is “started” is different per Type=. That distinction is the single most common reason a healthy-looking process still gets killed for timing out.

Common Causes

1. Type=notify service never sends READY=1. systemd waits for the process to call sd_notify(0, "READY=1") over the notification socket. If your code does not send it (or NotifyAccess= is too restrictive), systemd waits the full TimeoutStartSec and then kills a perfectly running process.
2. Hung or slow ExecStart. A real Type=simple/forking/oneshot workload that genuinely takes longer than 90s (large migration, cache warm-up, fsck) hits the limit.
3. Type=forking parent that never exits or writes no PID file. systemd considers a forking service started only when the original process forks and exits. If it stays in the foreground, the job hangs.
4. Device dependency timeout. A unit Requires=/After= a .device that never shows up (wrong UUID, unplugged disk, failed RAID member).
5. fstab mount waiting on a missing device. The classic emergency-mode boot: an entry in /etc/fstab references a UUID that no longer exists, so the device job times out and the mount fails.
6. TimeoutStartSec set too low for a service that legitimately needs more time.

How to Reproduce the Error

To see a Type=notify timeout safely on a test host, create a unit that claims to be notify but never sends readiness:

# /etc/systemd/system/timeout-demo.service  (example, do not deploy)
[Service]
Type=notify
ExecStart=/bin/sleep 600
TimeoutStartSec=15s

Because /bin/sleep never calls sd_notify, systemd waits 15 seconds, then terminates it and logs start operation timed out. Terminating. — exactly the production symptom, only faster.

For a device timeout, add a bogus UUID to /etc/fstab on a disposable VM and reboot; the boot will stall ~90s on Timed out waiting for device.

Diagnostic Commands

Start with the unit’s own status and logs. All commands below are read-only.

systemctl status myapp.service
journalctl -u myapp.service -b
journalctl -b -p err

Confirm how systemd thinks the unit should signal readiness, and what the timeout actually is:

systemctl show myapp.service -p Type -p TimeoutStartUSec -p NotifyAccess
systemctl show -p DefaultTimeoutStartUSec

Find which jobs and units are stuck or failed system-wide:

systemctl list-jobs
systemctl --failed

Pinpoint slow boot contributors and the dependency chain that is blocking:

systemd-analyze blame
systemd-analyze critical-chain myapp.service

A typical systemctl show output for a misconfigured notify service looks like this:

Type=notify
TimeoutStartUSec=1min 30s
NotifyAccess=main

TimeoutStartUSec=1min 30s confirms the default 90s window, and Type=notify tells you systemd is waiting for READY=1 that may never arrive.

systemd-analyze blame will surface a service that consumed almost exactly the timeout value:

1min 30.004s myapp.service
     12.881s NetworkManager-wait-online.service
      2.114s data.mount

When the problem is a device, journalctl -b -p err shows it plainly:

systemd[1]: dev-disk-by\x2duuid-4b1f...c92a.device: Job timed out.
systemd[1]: Timed out waiting for device /dev/disk/by-uuid/4b1f...c92a.
systemd[1]: Dependency failed for /mnt/data.

Step-by-Step Resolution

1. Identify the unit type and the result. From systemctl status, read the last Active: line and the result (timeout vs dependency). timeout points at the unit itself; dependency points upstream (usually a device or mount).

2. If it is a Type=notify service, verify readiness signalling. Either the application must call sd_notify once it is ready to serve, or the unit type is wrong. The application-side fix (C example) is to send readiness when the listening socket is up:

#include <systemd/sd-daemon.h>
/* call once the service is actually ready */
sd_notify(0, "READY=1");

If you cannot modify the program and it backgrounds itself correctly, switch the type. As a unit-file change (shown for reference):

[Service]
# was: Type=notify  -> use the type that matches the program's behaviour
Type=simple
# ensure NotifyAccess allows the right process when you DO use notify:
# NotifyAccess=all

3. If ExecStart is genuinely slow, raise the timeout rather than masking a hang. In a drop-in (/etc/systemd/system/myapp.service.d/override.conf):

[Service]
TimeoutStartSec=300

Use TimeoutStartSec=infinity only for one-shot maintenance units where unbounded runtime is acceptable.

4. For Type=forking, make sure the daemon actually forks and (ideally) writes a PID file, and that the unit declares PIDFile=. A foreground process under Type=forking will always time out.

5. For device/mount dependency failures, fix or relax the dependency. First confirm the device exists with read-only tools such as lsblk and blkid. If the UUID changed, update /etc/fstab to the correct one. For non-critical mounts that should never block boot, add nofail and a bounded device timeout:

# /etc/fstab (reference)
UUID=4b1f...c92a  /mnt/data  ext4  defaults,nofail,x-systemd.device-timeout=10s  0  2

nofail lets boot proceed if the device is absent; x-systemd.device-timeout=10s shortens the 90s wait to something tolerable.

6. Reload and retry. After editing unit files or fstab, systemd needs to re-read configuration. Run systemctl daemon-reload, then start the unit again and watch journalctl -u myapp.service -f to confirm it reaches active (running) before the timer.

Prevention and Best Practices

• Match Type= to program behaviour. Foreground process that never forks: Type=simple (or Type=exec). Self-backgrounding daemon: Type=forking with PIDFile=. Application that signals readiness: Type=notify. Short task that exits: Type=oneshot.
• Always send READY=1 for notify units, and set NotifyAccess= correctly (main, exec, or all) so the signalling process is allowed to talk to systemd.
• Set realistic TimeoutStartSec= per unit instead of bumping the global DefaultTimeoutStartSec=, which affects every service and can hide real hangs.
• Add nofail to optional mounts (backups, scratch, network shares) so a missing disk does not drop the box into emergency mode.
• Order dependencies explicitly with After=/Requires=/Wants= so services wait for the data they need rather than racing and stalling.
• Watch boot health over time with systemd-analyze blame and critical-chain; a service creeping toward its timeout is an early warning. For broader signal correlation, an automated monitoring and alerting workflow can flag units that repeatedly approach their start limits.

Related Errors

• Dependency failed for ... — almost always a downstream effect of a device or mount job that timed out first.
• Timed out waiting for device /dev/disk/by-uuid/... — the device-unit variant; check the UUID and physical/virtual disk presence.
• start operation timed out. Terminating. — the service-unit variant covered throughout this guide.
• Failed to start ... start request repeated too quickly — a different failure where the unit crashes fast and trips StartLimitBurst, not a timeout. See our companion guide on common systemd unit failures for that distinction.
• a stop job is running for ... — the shutdown-side mirror, governed by TimeoutStopSec=.

Frequently Asked Questions

Q: My service is clearly running, so why does systemd kill it for timing out? A: This is the signature of a Type=notify unit whose program never calls sd_notify(0, "READY=1"). systemd waits the full TimeoutStartSec for the readiness signal, decides startup failed, and terminates a process that was otherwise healthy. Either send READY=1 or change the type to match the program.

Q: What is the default timeout and where do I see it? A: The default is DefaultTimeoutStartSec=90s on most distributions. Check the global value with systemctl show -p DefaultTimeoutStartUSec and the per-unit value with systemctl show <unit> -p TimeoutStartUSec.

Q: Should I just set TimeoutStartSec=infinity? A: Only for trusted one-shot maintenance tasks. For long-running services, infinity removes a useful safety net — a genuinely hung unit will block dependent units forever. Set a generous but finite value instead.

Q: My boot hangs ~90 seconds then drops to emergency mode. What is wrong? A: An /etc/fstab entry references a device (usually by UUID) that no longer exists. The device job times out at 90s, the mount fails with dependency, and a critical mount sends you to emergency mode. Fix the UUID, or add nofail,x-systemd.device-timeout=10s for non-essential mounts.

Q: Where do I look first when I see a timeout? A: Run systemctl status <unit> and journalctl -u <unit> -b to read the result, then systemctl show <unit> -p Type -p TimeoutStartUSec -p NotifyAccess to confirm the type and limit. Those three commands isolate notify-readiness, slow ExecStart, and device problems in under a minute.