Hardening SSH on Linux Servers: A Practical Checklist

SSH is the front door to every Linux box you run, and the default config is built for compatibility, not security. The good news: a dozen lines in sshd_config eliminate the overwhelming majority of real-world SSH attacks. The bad news: get one wrong and you lock yourself out of a server you can’t physically reach.

After 25 years of running internet-facing fleets, here’s the hardening checklist I trust, the lockout-proof rollout, and where AI helps audit without making the call for you.

Rule zero: never test changes on a connection you can’t replace

Before touching sshd_config, open a second SSH session and keep it alive. Make your changes, restart sshd, and test login in a third session. If it fails, your second session is still in and you fix the config. Only close the safety session once a fresh login works. I have rescued myself with this rule more times than I’ll admit.

Disable password authentication

The single highest-impact change. Brute-force and credential-stuffing attacks die instantly when there’s no password to guess. In /etc/ssh/sshd_config:

PasswordAuthentication no
KbdInteractiveAuthentication no
PubkeyAuthentication yes

Obviously, set up your SSH key first:

ssh-copy-id user@host

Confirm key login works before you disable passwords, using the safety-session rule above.

Root should never log in directly. You log in as a user and sudo:

PermitRootLogin no

If automation genuinely needs root over SSH, use prohibit-password (key-only root) rather than yes, but a dedicated service account is better.

Limit who can log in

Whitelist explicitly:

AllowUsers deploy james
AllowGroups ssh-users

Anyone not listed can’t authenticate at all, regardless of valid keys. This is a quiet, powerful control most people skip.

Modern crypto and sane timeouts

Protocol 2
KexAlgorithms curve25519-sha256
Ciphers chacha20-poly1305@openssh.com,aes256-gcm@openssh.com
MaxAuthTries 3
LoginGraceTime 20
ClientAliveInterval 300
ClientAliveCountMax 2

MaxAuthTries 3 and LoginGraceTime 20 shrink the window for automated attacks. The cipher list drops legacy algorithms that scanners probe for.

Validate before you restart

This is the step that prevents lockouts. sshd can syntax-check the config without applying it:

sudo sshd -t

No output means it’s valid. Always run this before restarting. Then:

sudo systemctl reload sshd

reload over restart keeps existing sessions alive — another safety margin.

Let AI audit, but verify against `sshd -T`

This is where AI is genuinely useful: paste your full sshd_config and ask for a security review.

“Here’s my sshd_config. Identify weak settings, anything that increases attack surface, and any directive that could lock me out. Order findings by severity and give the exact line to change. Don’t recommend anything that would drop my current session.”

The model catches stale Ciphers lines, a forgotten PermitRootLogin yes, or a PasswordAuthentication yes buried in a Match block you forgot about. I keep this audit prompt in my Linux prompts.

But verify its findings against what sshd actually runs — config files have includes and Match overrides that change effective values:

sudo sshd -T | grep -i passwordauthentication

sshd -T prints the real, effective config. If AI says you’ve disabled passwords but sshd -T says otherwise, trust sshd -T. A Match block or an Include is overriding you. This is the same verify-don’t-trust discipline I apply during incident work.

Move beyond the config: fail2ban and a non-standard port

Config hardening removes the easy wins; the next layer slows the rest:

sudo apt install fail2ban

A basic jail bans IPs after repeated failures, which clears the brute-force noise from your logs. Changing the port off 22 is security-by-obscurity — it won’t stop a targeted attacker, but it dramatically cuts drive-by scanner volume, which is worth it just for cleaner logs.

For high-value hosts, go further and require a second factor. Match blocks let you apply rules selectively — for instance, requiring both a key and a one-time code for external addresses while leaving an internal management network on key-only:

Match Address 10.0.0.0/8
    AuthenticationMethods publickey
Match Address *,!10.0.0.0/8
    AuthenticationMethods publickey,keyboard-interactive

That AuthenticationMethods publickey,keyboard-interactive line forces both factors rather than accepting either one — a subtlety worth getting right, because a comma here means “and,” not “or.”

Don’t let AI weaken you “for compatibility”

One real caution: when something can’t connect, models often suggest re-enabling password auth or adding a legacy cipher to “fix” it. That’s almost always solving a key-permissions problem the wrong way. If a client can’t connect, debug with:

ssh -vvv user@host

The verbose output tells you whether it’s keys, ciphers, or AllowUsers — and the fix is usually adding the client properly, not weakening the server. Never let an AI suggestion roll back PasswordAuthentication no to make an error go away.

The minimal config worth copying

PermitRootLogin no
PasswordAuthentication no
KbdInteractiveAuthentication no
PubkeyAuthentication yes
AllowGroups ssh-users
MaxAuthTries 3
LoginGraceTime 20
ClientAliveInterval 300

The takeaway

Kill passwords, kill root login, whitelist users, modernize the ciphers — that’s 90% of the value. Always sshd -t before reload, keep a safety session open, and verify the live config with sshd -T. Let AI audit your file and flag the risky lines, but make every change yourself and confirm against the effective config. The front door deserves the extra two minutes.

AI security audits are assistive. Verify recommendations against sshd -T and your own threat model before applying.