🚨 Security Error Guide: 'dh key too small' TLS Handshake Failure After Hardening

Exact Error Message

After raising a client’s TLS security level, a connection to a server that still uses weak Diffie-Hellman parameters fails:

$ curl https://legacy.internal/health
curl: (35) error:0A00018A:SSL routines::dh key too small

openssl s_client reports the same handshake rejection in its own form:

140234:error:0A00018A:SSL routines:tls_process_ske_dhe:dh key too small:
ssl/statem/statem_clnt.c:2218:
sslv3 alert handshake failure

What the Error Means

In a TLS handshake using an ephemeral Diffie-Hellman (DHE) cipher suite, the server sends DH parameters whose size (in bits) determines the strength of the key exchange. Modern TLS libraries enforce a minimum DH size tied to their security level (SECLEVEL). “dh key too small” means the server offered DH parameters below the client’s required minimum (commonly 1024-bit params rejected by a client requiring 2048-bit or higher), so the client aborts the handshake.

This is a defensive control firing correctly: the hardened client refuses a weak key exchange that would be vulnerable to precomputation attacks. Engineers hit it after bumping OpenSSL’s SECLEVEL, upgrading to a newer TLS library, or enforcing a hardening profile — while the server side still ships 1024-bit DH params or a small built-in group. The fix is to strengthen the server’s DH parameters (or move it to ECDHE), not to weaken the client.

Common Causes

• 1024-bit (or smaller) server DH params. The server uses a legacy dhparam.pem or a default small group that the hardened client now rejects.
• Raised client SECLEVEL. A hardening change set OpenSSL SECLEVEL=2 (or higher), which mandates stronger DH/key sizes.
• Old server software with fixed groups. An appliance or legacy daemon offers only a small built-in DHE group with no way to enlarge it.
• DHE cipher preferred over ECDHE. The negotiated suite is DHE-based; switching to ECDHE avoids finite-field DH size limits entirely.
• Stale ssl_dhparam config. The server references an old 1024-bit params file generated years ago.
• Library upgrade raised the floor. A newer OpenSSL/GnuTLS version increased the default minimum DH size, breaking a previously-working peer.

How to Reproduce the Error

On a test host, generate a deliberately small DH params file, serve it, and connect from a client at a higher security level:

openssl dhparam -out /tmp/dh1024.pem 1024
# Point a test TLS server at dh1024.pem with a DHE-only cipher, then:
openssl s_client -connect localhost:8443 -cipher 'DHE' -cipherstrength 2 </dev/null

error:0A00018A:SSL routines:tls_process_ske_dhe:dh key too small
sslv3 alert handshake failure

Diagnostic Commands

These read-only checks confirm the DH size on the wire and the client’s required level.

# Connect and read the negotiated suite plus the server's DH params size
openssl s_client -connect legacy.internal:443 -tls1_2 </dev/null 2>/dev/null \
  | grep -iE 'Server Temp Key|Cipher|Protocol'

# Force a DHE suite to expose the finite-field DH size specifically
openssl s_client -connect legacy.internal:443 -cipher 'DHE' </dev/null 2>&1 \
  | grep -iE 'Server Temp Key|dh key too small|handshake failure'

# What security level does this client/OpenSSL enforce?
openssl version
openssl s_client -help 2>&1 | grep -i seclevel

# Inspect a server's DH params file size (on the server, read-only)
openssl dhparam -in /etc/ssl/dhparam.pem -text -noout 2>/dev/null | head -3

# Is the service even reachable on the expected port?
ss -tlnp 2>/dev/null | grep ':443'

# Check the server's effective cipher/dhparam config (nginx example)
grep -RinE 'ssl_dhparam|ssl_ciphers|ssl_ecdh_curve' /etc/nginx/ 2>/dev/null

The Server Temp Key line in openssl s_client output reveals the exact DH size (e.g. DH, 1024 bits), which is the smoking gun for this error.

Step-by-Step Resolution

1. Confirm the DH size on the wire. Run openssl s_client ... | grep 'Server Temp Key'. A line like Server Temp Key: DH, 1024 bits confirms the server is the weak side.

2. Prefer ECDHE over DHE. The most durable fix is to have the server offer ECDHE suites (e.g. with a modern curve), which avoids finite-field DH size limits entirely. Update the server’s cipher list and ssl_ecdh_curve.

3. If you must keep DHE, regenerate strong params on the server (2048-bit minimum, 4096 for long-lived services):

   sudo openssl dhparam -out /etc/ssl/dhparam.pem 2048

   Point the server at the new file (ssl_dhparam /etc/ssl/dhparam.pem; for nginx) and reload.

4. Verify the server config and reload safely so a typo does not take it offline:

   sudo nginx -t && sudo systemctl reload nginx

5. Do not lower the client. Avoid dropping SECLEVEL or re-enabling weak DHE to “fix” the error; that reintroduces the vulnerability the client is protecting against.

6. Re-test from the hardened client and confirm a strong key exchange:

   openssl s_client -connect legacy.internal:443 </dev/null 2>/dev/null | grep 'Server Temp Key'

   You should now see ECDH or DH, 2048 bits or larger.

Prevention and Best Practices

• Prefer ECDHE key exchange so you are not maintaining finite-field DH parameter files at all.
• Where DHE is required, generate at least 2048-bit dhparam and rotate it on a schedule; never reuse a 1024-bit file.
• Inventory legacy servers and appliances before raising client SECLEVEL so you upgrade the weak peers first.
• Use a managed TLS profile (modern cipher list, strong curves) applied consistently via configuration management.
• Periodically probe services with openssl s_client and check the Server Temp Key size to catch weak DH before clients do.

Related Errors

• no shared cipher / no cipher suites in common — an empty cipher overlap, a distinct handshake failure.
• unsupported protocol / wrong version number — a TLS version mismatch, not a DH size problem.
• ee key too small / ca md too weak — certificate (not DH) strength rejected by SECLEVEL, covered in the security hardening guides.
• sslv3 alert handshake failure — the generic alert that wraps several of these causes.

Frequently Asked Questions

Whose problem is “dh key too small” — client or server? The server: it offered weak DH parameters. The client is correctly refusing them. Fix the server’s DH params or move it to ECDHE.

Can I just lower my client’s security level? You can, but that reintroduces the weak key exchange. Strengthen the server instead.

How do I see the exact DH size? The Server Temp Key line from openssl s_client (e.g. DH, 1024 bits vs ECDH, X25519, 253 bits) shows it directly.

Why did this start after an OS upgrade? Newer OpenSSL/GnuTLS versions raised the default minimum DH size, so a server that was borderline before is now rejected.

Should I generate 4096-bit dhparam everywhere? 2048-bit meets modern requirements with less handshake cost; reserve 4096-bit for long-lived or high-assurance services. Better still, prefer ECDHE.