🏗️ Terraform Error: 'remote-exec provisioner error' timeout waiting for SSH connection

Exact Error Message

╷
│ Error: remote-exec provisioner error
│
│   with aws_instance.app,
│   on main.tf line 20, in resource "aws_instance" "app":
│   20:   provisioner "remote-exec" {
│
│ timeout - last error: dial tcp 203.0.113.10:22: i/o timeout
╵

You will also see the closely related connection refused form when the port is reachable but nothing is listening yet:

│ timeout - last error: SSH authentication failed (ubuntu@203.0.113.10:22):
│ dial tcp 203.0.113.10:22: connect: connection refused

What It Means

A remote-exec provisioner opens an SSH (or WinRM) session to the newly created resource and runs commands on that host. Before it can run anything it must establish the connection. Terraform retries the connection for the duration of the connection timeout (5 minutes by default), and if it never succeeds it aborts with timeout - last error: <last connection error>.

The trailing last error tells you which stage failed. i/o timeout means packets never got a response — usually a network/firewall block. connection refused means the host is reachable but sshd is not up yet. SSH authentication failed means you connected but the key or username was wrong. As with any provisioner failure during create, the resource is left tainted.

Common Causes

• A security group / firewall does not allow inbound TCP 22 (or 5985/5986 for WinRM) from the machine running Terraform.
• The instance has no public IP or route, and Terraform is not on the same private network / VPN.
• sshd has not finished starting when Terraform first connects (cloud-init still running) — connection refused.
• Wrong user, missing/incorrect private_key, or a key not baked into the image — authentication failed.
• The host in the connection block points at the private IP while Terraform runs outside the VPC.
• A too-short timeout on slow-booting images or large user-data scripts.

Diagnostic Commands

Test raw TCP reachability to the SSH port from the Terraform host:

nc -vz 203.0.113.10 22

Attempt the SSH connection manually with the same key and user Terraform uses:

ssh -i ~/.ssh/deploy.pem -o ConnectTimeout=10 ubuntu@203.0.113.10 'echo ok'

Inspect the security group rules attached to the instance:

aws ec2 describe-security-groups --group-ids sg-0abc123 \
  --query 'SecurityGroups[].IpPermissions'

Run apply with debug logging to see every connection retry and the final error:

TF_LOG=DEBUG terraform apply 2>&1 | tee apply-debug.log

Step-by-Step Resolution

1. Read the last error. i/o timeout → network/firewall; connection refused → sshd not ready; authentication failed → credentials. Reproduce with the matching diagnostic above.

2. If it is a firewall problem, open the port from the correct source and make sure the instance has a reachable address. Confirm the connection block targets the right host:

resource "aws_instance" "app" {
  ami                    = var.ami_id
  instance_type          = "t3.small"
  key_name               = aws_key_pair.deploy.key_name
  vpc_security_group_ids = [aws_security_group.ssh.id]

  provisioner "remote-exec" {
    inline = ["cloud-init status --wait", "sudo systemctl restart app"]

    connection {
      type        = "ssh"
      host        = self.public_ip
      user        = "ubuntu"
      private_key = file("~/.ssh/deploy.pem")
      timeout     = "10m"
    }
  }
}

3. For connection refused right after boot, wait for the host to be ready rather than lengthening blindly — but do raise timeout for slow images, and let cloud-init settle (cloud-init status --wait as the first inline command).

4. For authentication failed, verify the user matches the AMI (ubuntu, ec2-user, admin, etc.), the private_key matches the injected public key, and the key is not passphrase-protected (Terraform cannot prompt).

5. If Terraform runs outside the VPC, either give the instance a public IP, connect through a bastion_host in the connection block, or run Terraform from inside the network / over VPN.

6. Because the failed provisioner tainted the resource, re-apply to recreate and retry:

terraform apply

7. Confirm the plan replaces the tainted resource and the provisioner completes cleanly this time.

Prevention

• Prefer user_data/cloud-init or a configuration-management tool over remote-exec; it removes the SSH dependency entirely.
• Always begin inline commands with cloud-init status --wait so you do not race the boot process.
• Set a realistic timeout (10m+) for images with heavy first-boot work.
• Scope security groups to the Terraform runner’s address rather than opening 22 to the world.
• Use a bastion_host in the connection block for private instances instead of assigning public IPs.
• Keep the SSH key passphrase-free and confirm the user matches the AMI’s default account.

Related Errors

• local-exec provisioner error: exit status N — the local variant that fails on command exit code, not connectivity.
• SSH authentication failed — you reached the host but the key/user was rejected.
• Failed to read ssh private key: no key found — a bad private_key path or format.
• Resource is tainted — the state condition left after any provisioner failure.

Frequently Asked Questions

What does timeout - last error: i/o timeout actually mean? Terraform’s TCP packets to port 22 got no response for the whole connection timeout window — almost always a security-group or routing block, not an SSH-level problem.

How do I stop racing cloud-init? Make the first inline command cloud-init status --wait, and raise the connection timeout so Terraform keeps retrying while the host finishes booting.

Why does connection refused differ from i/o timeout? Refused means the host is reachable but nothing is listening on the port yet (sshd still starting); timeout means the packets never reached a listening host at all.

Can I reach a private instance without a public IP? Yes — set a bastion_host (and its credentials) in the connection block, or run Terraform from inside the VPC over VPN. For more connectivity patterns, see the prompt library.

Why is the instance recreated after the timeout? A provisioner failure during create taints the resource, so the next apply destroys and recreates it. Fix connectivity first, then re-apply. More fixes are in the Terraform guides.