🚨 TLS & Security Error Guide: 'tls: handshake failure' / no cipher suites in common

Overview

A TLS handshake failure means the client and server could not agree on the parameters needed to establish a secure session — most often a protocol version or cipher suite that both sides support. The server (or client) sends a fatal handshake_failure alert (alert 40) and the connection is torn down before any application data flows. The phrase “no cipher suites in common” is the specific case where the negotiated protocol is fine but the cipher/curve/signature lists do not overlap.

You will see variants depending on the stack:

tls: handshake failure

error:0A000410:SSL routines::sslv3 alert handshake failure

no cipher suites in common

These are negotiation failures, not certificate-trust failures (which report “unable to get local issuer” or “expired”). The handshake never reached certificate verification — the two sides could not even pick a protocol/cipher. The usual triggers are a TLS 1.0/1.1 client hitting a TLS 1.2+-only server, a hardened cipher list with no overlap, or a key type (RSA vs ECDSA) the peer cannot use.

Symptoms

• Connection fails immediately with “handshake failure” / “handshake_failure” / “no cipher suites in common”.
• It fails before any certificate prompt or hostname check.
• An older client (legacy Java, old curl/OpenSSL, IoT device) fails while modern clients succeed.
• The failure started right after a server TLS-hardening change.

echo | openssl s_client -connect api.internal.example.com:443 -servername api.internal.example.com 2>&1 | head -5

CONNECTED(00000003)
40D7F0...:error:0A000410:SSL routines::sslv3 alert handshake failure:ssl/record/rec_layer_s3.c:...
---
no peer certificate available
---

no peer certificate available confirms the handshake died before the server even sent its cert.

Common Root Causes

1. Protocol-version mismatch (client too old or server too strict)

The server requires TLS 1.2/1.3 but the client only offers TLS 1.0/1.1.

echo | openssl s_client -connect api.internal.example.com:443 -tls1_1 2>&1 | grep -E 'alert|Protocol|Cipher'
echo | openssl s_client -connect api.internal.example.com:443 -tls1_2 2>&1 | grep -E 'alert|Protocol|Cipher'

alert handshake failure
    Protocol  : TLSv1.2
    Cipher    : ECDHE-RSA-AES256-GCM-SHA384

TLS 1.1 fails while TLS 1.2 works — the server dropped legacy protocols.

2. No overlapping cipher suite

Both sides support TLS 1.2 but the server’s allowed ciphers and the client’s offered ciphers do not intersect.

nmap --script ssl-enum-ciphers -p 443 api.internal.example.com 2>/dev/null | grep -A20 'TLSv1.2'

|       TLSv1.2:
|         ciphers:
|           TLS_ECDHE_ECDSA_WITH_AES_256_GCM_SHA384 (secp256r1) - A

If the server only offers ECDSA suites and the client only speaks RSA suites, there is no common cipher.

3. Certificate key type the peer cannot use (RSA vs ECDSA)

The server has only an ECDSA certificate, but the client offers only RSA-key-exchange ciphers (or vice versa).

echo | openssl s_client -connect api.internal.example.com:443 2>/dev/null \
  | openssl x509 -noout -text | grep -E 'Public Key Algorithm|ASN1 OID'

            Public Key Algorithm: id-ecPublicKey
                ASN1 OID: prime256v1

An ECDSA-only cert plus an RSA-only client = no usable cipher suite, hence “no cipher suites in common”.

4. No overlapping elliptic curve / signature algorithm (TLS 1.3)

In TLS 1.3 the handshake can fail if the client’s supported_groups (curves) or signature algorithms do not include any the server accepts.

openssl s_client -connect api.internal.example.com:443 -tls1_3 -groups X25519 2>&1 | grep -E 'alert|Negotiated'
openssl s_client -connect api.internal.example.com:443 -tls1_3 -groups secp384r1 2>&1 | grep -E 'alert|Negotiated'

alert handshake failure
Negotiated TLS1.3 group: secp384r1

X25519 is rejected but secp384r1 works — the server’s curve allow-list is narrow.

5. SNI required but missing/wrong

A server hosting multiple certs returns a handshake failure (rather than a default cert) when no SNI or an unknown SNI is sent.

echo | openssl s_client -connect 10.0.0.5:443 2>&1 | grep -E 'alert|subject'
echo | openssl s_client -connect 10.0.0.5:443 -servername api.internal.example.com 2>&1 | grep -E 'alert|subject'

alert handshake failure
subject=CN = api.internal.example.com

Without SNI the vhost has no cert to present and aborts.

6. Client cert required (mTLS) but none presented

The server requests a client certificate and fails the handshake when the client offers none or an unaccepted one.

echo | openssl s_client -connect api.internal.example.com:8443 2>&1 | grep -E 'alert|certificate request|CertificateRequest'

alert handshake failure

echo | openssl s_client -connect api.internal.example.com:8443 \
  -cert client.crt -key client.key 2>&1 | grep -E 'Verify return code|alert'

    Verify return code: 0 (ok)

Supplying the client cert clears the failure — it was an mTLS requirement.

Diagnostic Workflow

Step 1: Confirm it is negotiation, not trust

echo | openssl s_client -connect <HOST>:443 -servername <HOST> 2>&1 | grep -E 'alert handshake failure|no peer certificate|Cipher'

“alert handshake failure” + “no peer certificate available” = negotiation failure (this guide). A cert that is shown but rejected is a different (trust) problem.

Step 2: Bisect the protocol version

for v in tls1 tls1_1 tls1_2 tls1_3; do
  printf '%s: ' "$v"
  echo | openssl s_client -connect <HOST>:443 -$v 2>&1 | grep -qE 'Cipher *: .+[^ ]' && echo OK || echo FAIL
done

The lowest version that returns OK tells you the server’s floor; if your client is below it, that is the cause.

Step 3: Enumerate the server’s accepted ciphers and key type

nmap --script ssl-enum-ciphers -p 443 <HOST> 2>/dev/null | grep -E 'TLSv1|ciphers:|TLS_'
echo | openssl s_client -connect <HOST>:443 2>/dev/null | openssl x509 -noout -text | grep 'Public Key Algorithm'

Compare the server’s cipher families (ECDSA vs RSA) to what the client can offer.

Step 4: Test curves and SNI explicitly (TLS 1.3 / multi-vhost)

openssl s_client -connect <HOST>:443 -tls1_3 -groups X25519:secp256r1:secp384r1 2>&1 | grep -E 'Negotiated|alert'
echo | openssl s_client -connect <HOST>:443 -servername <HOST> 2>&1 | grep -E 'subject|alert'

Step 5: Reconcile policy on both ends

# server (nginx) - widen only as policy allows
grep -E 'ssl_protocols|ssl_ciphers|ssl_ecdh_curve' /etc/nginx/nginx.conf
# then reload and re-test
sudo nginx -t && sudo systemctl reload nginx
echo | openssl s_client -connect <HOST>:443 2>&1 | grep -E 'Protocol|Cipher'

Example Root Cause Analysis

After a security review enabled a modern cipher policy on the API gateway, a legacy Java 8 batch client began failing every call with:

javax.net.ssl.SSLHandshakeException: Received fatal alert: handshake_failure

Modern clients (curl, browsers) work fine, which points at the client’s capabilities, not the cert. Probing the server’s protocol floor:

for v in tls1_1 tls1_2 tls1_3; do printf '%s: ' "$v"; \
  echo | openssl s_client -connect gw.internal.example.com:443 -$v 2>&1 \
  | grep -qE 'Cipher *: .+[^ ]' && echo OK || echo FAIL; done

tls1_1: FAIL
tls1_2: OK
tls1_3: OK

The server now requires TLS 1.2+. Enumerating ciphers shows it offers only ECDHE-ECDSA suites:

nmap --script ssl-enum-ciphers -p 443 gw.internal.example.com 2>/dev/null | grep TLS_

|           TLS_ECDHE_ECDSA_WITH_AES_256_GCM_SHA384 (secp256r1) - A

The old Java 8 default config negotiated TLS 1.1 and lacked the ECDSA suites the gateway now requires — no protocol and no cipher overlap. The correct fix is on the client: enable TLS 1.2 and modern ciphers (-Dhttps.protocols=TLSv1.2 -Djdk.tls.client.cipherSuites=...) or upgrade the JRE, rather than weakening the gateway.

# after enabling TLS1.2 on the client JVM:
handshake succeeded, HTTP 200

The legacy client connects without lowering the gateway’s hardened policy.

Prevention Best Practices

• Treat TLS-hardening changes as a coordinated rollout: inventory client TLS versions/ciphers before raising the server floor so legacy clients fail in a test window, not production.
• Provision both an ECDSA and an RSA certificate on servers that must serve mixed/legacy clients, so cipher overlap is never the issue.
• Keep ssl_protocols/ssl_ciphers/ssl_ecdh_curve in version control and align them with a published baseline rather than ad-hoc edits.
• Fix the client (upgrade runtime, enable TLS 1.2/1.3) instead of weakening server policy whenever feasible. See the security hardening guides for a recommended cipher/protocol baseline.
• For an mTLS endpoint, document that a client cert is required so “handshake failure” is not mistaken for a server problem.
• When a hardening change triggers an outage, the free incident assistant can distinguish a protocol-version vs cipher-overlap vs mTLS cause from the s_client and ssl-enum-ciphers output.

Quick Command Reference

# Negotiation vs trust?
echo | openssl s_client -connect <HOST>:443 -servername <HOST> 2>&1 | grep -E 'alert handshake failure|no peer certificate'

# Bisect protocol versions
for v in tls1 tls1_1 tls1_2 tls1_3; do printf '%s: ' "$v"; \
  echo | openssl s_client -connect <HOST>:443 -$v 2>&1 | grep -qE 'Cipher *: .+[^ ]' && echo OK || echo FAIL; done

# Server's accepted ciphers and cert key type
nmap --script ssl-enum-ciphers -p 443 <HOST> 2>/dev/null | grep -E 'TLSv1|TLS_'
echo | openssl s_client -connect <HOST>:443 2>/dev/null | openssl x509 -noout -text | grep 'Public Key Algorithm'

# TLS 1.3 curves and SNI
openssl s_client -connect <HOST>:443 -tls1_3 -groups X25519:secp256r1:secp384r1 2>&1 | grep -E 'Negotiated|alert'
echo | openssl s_client -connect <HOST>:443 -servername <HOST> 2>&1 | grep -E 'subject|alert'

# mTLS test
echo | openssl s_client -connect <HOST>:8443 -cert client.crt -key client.key 2>&1 | grep -E 'Verify return code|alert'

Conclusion

A tls: handshake failure / “no cipher suites in common” is a negotiation failure that dies before certificate verification. Work the parameters both sides offer:

1. Protocol-version mismatch — an old client below the server’s TLS floor (or vice versa).
2. No overlapping cipher suite between the client’s and server’s lists.
3. A certificate key type (RSA vs ECDSA) the peer’s ciphers cannot use.
4. No common elliptic curve or signature algorithm in TLS 1.3.
5. SNI required but missing or unknown, so the vhost has no cert to present.
6. mTLS required but the client presented no (or an unaccepted) client certificate.

Bisect the protocol version, enumerate ciphers/curves, and reconcile both ends — and prefer upgrading the client to weakening a hardened server policy.