strace's blog

Diagnosing SSL/TLS Handshake Failures

Debugging

SSL/TLS handshake failures are among the most frustrating issues to debug. The error messages are often opaque ("sslv3 alert certificate unknown"), and the handshake itself involves a dozen negotiation steps that can fail at any point.

This guide covers the repeatable diagnostic approach I use.

1. The Swiss Army Knife: openssl s_client

openssl s_client is the single most valuable TLS debugging tool:

# Basic connection test
openssl s_client -connect example.com:443

# Show full certificate chain
openssl s_client -connect example.com:443 -showcerts

# Debug a specific TLS version
openssl s_client -connect example.com:443 -tls1_2
openssl s_client -connect example.com:443 -tls1_3

# Specify SNI (critical for multi-tenant hosts)
openssl s_client -connect 1.2.3.4:443 -servername example.com

What the output tells you:

  • CONNECTED(00000003) — TCP connection succeeded
  • verify error:num=... — certificate validation failure
  • SSL-Session: block — negotiated cipher, protocol version
  • --- — clean closure means the full handshake completed

2. Certificate Chain Problems

The most common class of TLS failure:

# Verify the full chain
openssl s_client -connect example.com:443 -showcerts -verify_return_error

# Download each certificate manually
openssl s_client -connect example.com:443 -showcerts 2>/dev/null </dev/null |   awk '/^-----BEGIN CERTIFICATE-----/,/^-----END CERTIFICATE-----/' > chain.pem

# Verify the chain locally
openssl verify -CApath /etc/ssl/certs chain.pem

Common chain issues:

  • Missing intermediate cert — the server only sends the leaf certificate. Fix: configure the full chain on the server.
  • Wrong intermediate — the server sends an intermediate that doesn't chain to the leaf. Regenerate with the correct CA.
  • Expired intermediate — check the validity period of each cert in the chain.
  • Self-signed root in chain — roots should NOT be sent by the server; the client trusts them locally.

3. Cipher Suite Mismatch

When client and server can't agree on a cipher:

# List ciphers supported by your openssl version
openssl ciphers -v 'ALL:COMPLEMENTOFALL'

# Test a specific cipher
openssl s_client -connect example.com:443 -cipher 'ECDHE-RSA-AES256-GCM-SHA384'

# See which ciphers the server advertises
openssl s_client -connect example.com:443 </dev/null 2>/dev/null |   openssl ciphers -V 'ALL' | head -20

Diagnostic approach:

  1. Check the server's minimum TLS version (TLS 1.0/1.1 ciphers are different from 1.3)
  2. Check if the client's cipher list is restricted (Java jdk.tls.disabledAlgorithms, Go crypto/tls.Config.CipherSuites)
  3. Test with a modern client to isolate whether it's a server or client configuration issue

4. Protocol Version Negotiation

# Test each protocol version explicitly
for ver in tls1 tls1_1 tls1_2 tls1_3; do
  echo "=== $ver ==="
  openssl s_client -connect example.com:443 -$ver </dev/null 2>&1 |     grep -E '(CONNECTED|verify error|SSL handshake)'
done

Common version issues:

  • Server only supports TLS 1.2, client requires 1.3
  • Server has TLS 1.0/1.1 enabled (security audit finding)
  • Middlebox (F5, AWS NLB) strips TLS 1.3 ClientHello due to byte 44 issue

5. Debugging with Wireshark/TShark

When OpenSSL succeeds but your application fails:

# Capture the handshake
tshark -i eth0 -f "tcp port 443" -w handshake.pcap -a duration:30

# Filter for TLS handshake packets
tshark -r handshake.pcap -Y "tls.handshake"

# View ClientHello and ServerHello details
tshark -r handshake.pcap -Y "tls.handshake.type == 1 || tls.handshake.type == 2"   -T fields -e tls.handshake.ciphersuite -e tls.handshake.extensions_server_name

What to look for in the capture:

  • ClientHello → ServerHello round-trip time (high latency = network issue)
  • Certificate size (large certs > 8KB can cause fragmentation)
  • ServerHello has no cipher overlap with ClientHello

6. OCSP Stapling

# Check if the server staples OCSP responses
openssl s_client -connect example.com:443 -status </dev/null 2>&1 |   grep -A 20 'OCSP response:'

# A successful response looks like:
# OCSP Response Status: successful (0x0)
# Response Type: Basic OCSP Response

Missing OCSP stapling can cause slow page loads as the browser fetches revocation status separately.

7. Application-Layer TLS Issues

Sometimes TLS works mechanically but the application rejects the connection:

# Check SNI mismatch (server returns default cert)
openssl s_client -connect example.com:443 -servername wrong.example.com

# Check ALPN (Application-Layer Protocol Negotiation)
openssl s_client -connect example.com:443 -alpn http/1.1 </dev/null 2>&1 |   grep -i 'alpn'

# Check for client certificate requirements
# (server sends CertificateRequest during handshake)
openssl s_client -connect example.com:443 -state 2>&1 | grep 'CertificateRequest'

Diagnostic Decision Tree

Handshake fails?
├── openssl s_client succeeds?
│   ├── Yes → Problem is in your application's TLS stack
│   │   └── Compare cipher lists, CA bundles, TLS versions
│   └── No → Server-side issue
│       ├── Certificate error?
│       │   └── Check chain, expiry, hostname match
│       ├── Protocol error?
│       │   └── Test each TLS version, check SNI
│       └── Connection timeout?
│           └── Firewall, load balancer, or routing issue

The key insight: always start with openssl s_client. If it connects, your application is the problem. If it fails, the server or network is the problem. Don't cross the streams.