SSH With Kali Linux for DevOps Engineers

SSH (Secure Shell) is the everyday tool for administering remote Linux hosts. From your Kali workstation you’ll use it to log into servers, run commands, copy files, and automate deployments — all over an encrypted connection. This lesson teaches SSH from an administration perspective: connecting to hosts you own or are authorized to manage, not gaining access to systems you don’t.

What SSH Is and Why It Matters

SSH gives you an encrypted, authenticated channel to a remote machine’s shell. Instead of standing in front of a server, you open a terminal on your Kali box and get a shell on a host that might be in another rack, region, or cloud.

Two pieces make it work:

Component	Role
sshd (SSH server/daemon)	Runs on the remote host, listens on TCP port 22 by default
ssh (SSH client)	Runs on your Kali machine, initiates the connection

Kali ships with the OpenSSH client (ssh) installed. The server (openssh-server) is usually stopped by default so your workstation isn’t accepting inbound logins — which is what you want for a laptop.

🛠️ DevOps Perspective — SSH is the backbone of remote administration and automation. Ansible, rsync, git over SSH, CI/CD deploy steps, and scp in scripts all ride on top of it. Getting keys and config right once saves you from typing passwords and hostnames for the rest of your career.

The SSH Client: Connecting to a Host

The basic form is ssh user@host:

ssh devops@192.168.1.50

This connects as the user devops to the host at 192.168.1.50. If the remote user matches your local username you can omit it:

ssh 192.168.1.50

You can also connect by hostname and choose a non-default port with -p:

ssh -p 2222 devops@server.internal.example.com

Run a single command remotely instead of opening an interactive shell by appending it:

ssh devops@192.168.1.50 'uptime && df -h'

The first time you connect to a new host, SSH shows the server’s host key fingerprint and asks you to confirm:

The authenticity of host '192.168.1.50' can't be established.
ED25519 key fingerprint is SHA256:aZ9....
Are you sure you want to continue connecting (yes/no/[fingerprint])?

Typing yes records the key in ~/.ssh/known_hosts. On future connections SSH checks that the host still presents the same key — this is how it detects that you’re talking to the same machine and not an impostor.

Public and Private Keys

Password logins work, but key-based authentication is more secure and the standard for automation. It uses a key pair:

• Private key — stays on your Kali machine, never leaves it. This is the secret.
• Public key — copied to each server you want to log into. Safe to share.

The server keeps your public key in its ~/.ssh/authorized_keys file. When you connect, the two keys perform a cryptographic handshake that proves you hold the private key — without ever sending it over the network.

Generating a key pair with ssh-keygen

ssh-keygen -t ed25519 -C "devops@kali"

• -t ed25519 chooses the modern Ed25519 algorithm — fast, short, and strong. (Use -t rsa -b 4096 only if you must support older systems.)
• -C "devops@kali" adds a comment so you can identify the key later.

You’ll be prompted for a file location (default ~/.ssh/id_ed25519) and a passphrase. Set a passphrase — it encrypts the private key on disk, so a stolen key file is useless without it.

This produces two files:

File	Contents
~/.ssh/id_ed25519	Your private key (keep secret)
~/.ssh/id_ed25519.pub	Your public key (copy to servers)

Installing your key with ssh-copy-id

The easy way to put your public key on a server is ssh-copy-id:

ssh-copy-id devops@192.168.1.50

It logs in once (using your password) and appends your public key to the remote ~/.ssh/authorized_keys, fixing permissions along the way. After this, ssh devops@192.168.1.50 logs you in with your key — no password prompt.

If you keep multiple keys, point it at the right one:

ssh-copy-id -i ~/.ssh/id_ed25519.pub devops@192.168.1.50

🔐 Security Note — Prefer key-based authentication over passwords: keys resist brute-force and credential-stuffing attacks that passwords do not. Never share, email, or commit your private key — only the .pub file is meant to travel. Always set a passphrase on the private key, and once keys work, consider disabling password logins on your servers (PasswordAuthentication no in sshd_config).

File Permissions: The Rule That Trips Everyone Up

OpenSSH refuses to use keys that are too readable, because a world-readable private key is a security hole. Get these exactly right:

Path	Permission	Meaning
~/.ssh	700 (drwx------)	Only you can access the directory
~/.ssh/id_ed25519 (private key)	600 (-rw-------)	Only you can read/write
~/.ssh/id_ed25519.pub (public key)	644	World-readable is fine
~/.ssh/authorized_keys (on server)	600	Only the owner can modify
~/.ssh/config	600	Protects your connection settings

Set them like this:

chmod 700 ~/.ssh
chmod 600 ~/.ssh/id_ed25519
chmod 644 ~/.ssh/id_ed25519.pub

If ownership is wrong, fix it too — the files must be owned by your user, not root. See Users, Permissions & sudo for how chmod, chown, and octal permission bits work.

The ~/.ssh/config File

Typing full connection strings gets old fast. A per-user config file lets you define short aliases with all the details baked in:

# ~/.ssh/config

Host web1
    HostName 192.168.1.50
    User devops
    Port 22
    IdentityFile ~/.ssh/id_ed25519

Host bastion
    HostName bastion.example.com
    User admin

# Reach a private host through the bastion in one command
Host db1
    HostName 10.0.5.20
    User devops
    ProxyJump bastion

Now ssh web1 expands to the full devops@192.168.1.50 connection, and ssh db1 transparently hops through the bastion. This is where SSH stops being a command you type and becomes infrastructure you maintain.

🛠️ DevOps Perspective — A well-kept ~/.ssh/config is living documentation of your fleet. Host aliases show up as tab-completion targets and make scp, rsync, and Ansible inventories cleaner. ProxyJump (bastion hosts) keeps private-subnet machines reachable without exposing them to the internet.

Copying Files: SCP and SFTP

You’ll constantly move files between your Kali box and remote hosts. Two tools cover it, both over the same encrypted SSH channel.

scp — copy files like cp, but remote

# Local -> remote
scp ./app.tar.gz devops@192.168.1.50:/tmp/

# Remote -> local
scp devops@192.168.1.50:/var/log/nginx/error.log ./

# Recursively copy a directory
scp -r ./config devops@192.168.1.50:/etc/myapp/

If you have a ~/.ssh/config alias, it works here too: scp ./app.tar.gz web1:/tmp/.

sftp — an interactive file-transfer session

sftp devops@192.168.1.50

This drops you into an interactive prompt where you navigate and transfer files:

sftp> ls
sftp> cd /var/www
sftp> get index.html        # download to local
sftp> put newpage.html      # upload from local
sftp> bye

Use scp for quick one-off copies in scripts; use sftp when you want to browse and move several files interactively. For syncing whole directory trees efficiently, rsync -avz -e ssh is the heavier-duty option once you’re comfortable.

🧪 Try It — Spin up a second Linux VM (or a container you control) as your “server.” From Kali: (1) ssh-keygen -t ed25519 to make a key, (2) ssh-copy-id user@server-ip to install it, (3) ssh user@server-ip and confirm you log in with no password, (4) add a Host alias in ~/.ssh/config and connect using just the alias, (5) scp a test file over and verify it landed. Only practice against machines you own or are explicitly authorized to use.

Troubleshooting SSH

When a connection misbehaves, work through these in order.

Turn on verbose output

The single most useful debugging flag is -v (add more vs for more detail):

ssh -v devops@192.168.1.50

-v, -vv, and -vvv print each step of the handshake: which keys were offered, which authentication methods were tried, and exactly where it failed. Read the last few lines before the failure — they almost always name the cause.

Common failures

Symptom	Likely cause	Fix
Connection refused	sshd not running, or wrong port	Confirm the daemon is up and the port; check firewalls/security groups
Connection timed out	Network/firewall blocking, wrong IP	Verify reachability with ping/nc; check routing and security groups
Permission denied (publickey)	Key not installed, wrong key, or bad permissions	See the tip below
Host key verification failed	Server’s key changed since last time	See “host key changes” below
Repeated password prompts despite a key	Wrong IdentityFile, or agent not offering the key	Check ~/.ssh/config and ssh-add -l

🔎 Troubleshooting Tip — Permission denied (publickey) means the server rejected your key. Walk through: (1) Is your public key actually in the server’s ~/.ssh/authorized_keys? Re-run ssh-copy-id user@host if unsure. (2) Are permissions correct — 700 on ~/.ssh, 600 on the private key locally and on authorized_keys remotely? OpenSSH silently ignores keys with loose permissions. (3) Are you offering the right key? Run ssh -v user@host and look for Offering public key: lines. (4) Is the right identity loaded — ssh-add -l? Force a specific key with ssh -i ~/.ssh/id_ed25519 user@host.

Host key changes

If you see a loud warning like REMOTE HOST IDENTIFICATION HAS CHANGED!, the host is presenting a different key than the one stored in known_hosts. This is expected after a legitimate rebuild, reinstall, or IP reassignment — but it can also indicate a man-in-the-middle situation, so verify the change is expected before proceeding.

Once you’ve confirmed it’s a legitimate change, remove the stale entry:

ssh-keygen -R 192.168.1.50

Your next connection will prompt you to accept the new key.

The SSH agent

The agent holds your decrypted private key in memory so you enter the passphrase once per session instead of on every connection:

eval "$(ssh-agent -s)"     # start the agent
ssh-add ~/.ssh/id_ed25519  # load your key (prompts for passphrase once)
ssh-add -l                 # list loaded keys

If git push or ssh keeps asking for your passphrase, the agent probably isn’t running or your key isn’t loaded — ssh-add -l tells you in one line.

Where This Fits

SSH sits on top of the TCP/IP fundamentals you’ll want fresh — port 22, name resolution, and reachability all matter when a connection fails. Review Networking Fundamentals if Connection refused or timed out errors are common for you. And because SSH is fundamentally about the right user having the right access to the right files, Users, Permissions & sudo is essential background. For broader remote-administration topics, browse the Linux Admins guides.

What You Learned

• SSH gives you an encrypted, authenticated shell on remote hosts; the ssh client connects to the sshd server, and Kali ships the client ready to use.
• Key-based authentication (ssh-keygen to create a pair, ssh-copy-id to install the public key) is more secure than passwords and is the foundation for automation — keep the private key secret and passphrase-protected.
• Correct permissions are non-negotiable: 700 on ~/.ssh, 600 on private keys and authorized_keys; OpenSSH ignores keys that are too readable.
• ~/.ssh/config turns long connection strings into short, maintainable host aliases, including bastion hops with ProxyJump.
• scp and sftp move files over the same secure channel — scp for scripted one-offs, sftp for interactive browsing.
• Troubleshoot with ssh -v, resolve Permission denied (publickey) by checking key installation and permissions, clear changed host keys with ssh-keygen -R, and use the SSH agent to avoid repeated passphrase prompts.