Debugging Docker Container Networking

Docker networking is elegant when it works and maddening when it doesn't. The complexity comes from the layers: Linux bridge, iptables rules, network namespaces, and Docker's embedded DNS server all interact in non-obvious ways.

Here's my systematic approach to untangling container networking issues.

1. The Three Connection States

Container connectivity issues fall into three categories:

Symptom Likely Cause Quick Test
connection refused Service not listening or wrong port docker exec <c> ss -tlnp
connection timed out Firewall / routing / no route docker exec <c> ping -c 2 <target>
no route to host Network misconfiguration docker network inspect <net>

2. Diagnosing "Connection Refused"

This is the most misleading error. "Refused" means the TCP SYN reached the target but nothing accepted the connection:

# Inside the target container: what's actually listening?
docker exec <target> ss -tlnp

# Expected output:
# State   Recv-Q  Send-Q  Local Address:Port   Peer Address:Port  Process
# LISTEN  0       128     0.0.0.0:8080         0.0.0.0:*          users:(("node",pid=1,fd=3))

# Common mistake: service bound to 127.0.0.1 inside the container
# This is NOT accessible from outside the container!
# Fix: bind to 0.0.0.0 or :: instead

The 127.0.0.1 trap: If your app inside the container binds to 127.0.0.1 (localhost), it's only reachable from within the same network namespace. Docker's port mapping (-p 8080:8080) won't help. Always bind to 0.0.0.0 in containerized apps.

3. Diagnosing "Connection Timed Out"

A timeout means the SYN was sent but no SYN-ACK came back:

# Step 1: Can you reach the target at all?
docker exec <source> ping -c 2 <target-ip>

# Step 2: Trace the route
docker exec <source> traceroute -n <target-ip>

# Step 3: Check iptables rules on the host
sudo iptables -L -n -t filter
sudo iptables -L -n -t nat

# Step 4: Check Docker's iptables chains specifically
sudo iptables -L DOCKER -n -t filter
sudo iptables -L DOCKER -n -t nat

Common timeout causes:

  • Host firewall (iptables/nftables) dropping forwarded packets — check iptables -L FORWARD
  • Docker's --icc=false (inter-container communication disabled)
  • Custom --iptables=false on the Docker daemon

4. Network Namespace Deep Dive

Every container has its own network namespace. You can inspect it from the host:

# Find the container's PID
docker inspect <container> --format '{{.State.Pid}}'

# Enter the container's network namespace
PID=$(docker inspect <container> --format '{{.State.Pid}}')
sudo nsenter -t $PID -n ip addr
sudo nsenter -t $PID -n ip route
sudo nsenter -t $PID -n iptables -L -n

# Check the veth pair on the host
ip link show | grep veth
ethtool -S vethXXXXXX  # shows per-interface stats

Key things to check in the namespace:

  • Does the container have a default route? (ip route | grep default)
  • Is the DNS server reachable? (cat /etc/resolv.conf, nslookup google.com)
  • Is the correct interface up? (ip link show)

5. Docker DNS Debugging

Docker runs an embedded DNS server at 127.0.0.11 inside each "bridge" or custom network container:

# Query Docker DNS directly
docker exec <container> nslookup <service-name>
docker exec <container> nslookup <other-container-name>

# Check the DNS configuration
docker exec <container> cat /etc/resolv.conf

# Bypass Docker DNS entirely
docker run --dns 8.8.8.8 alpine nslookup google.com

Common DNS issues:

  • Container on different networks can't resolve each other by name
  • docker compose creates a network named <project>_default — containers in a different compose project can't resolve each other
  • Search domains configured in /etc/resolv.conf shadow public domains (e.g., a search domain of .com breaks google.com)

6. Port Publishing Debugging

When you can reach a container from other containers but not from the host:

# Check what ports are published
docker port <container>

# Check iptables NAT rules
sudo iptables -t nat -L DOCKER -n

# Check if the host is listening
ss -tlnp | grep <host-port>

# Test with curl from the host
curl -v http://localhost:<host-port>
curl -v http://127.0.0.1:<host-port>
curl -v http://<docker-host-ip>:<host-port>

The 127.0.0.1 vs 0.0.0.0 distinction: If Docker is binding to 127.0.0.1:<port>, it's only accessible from the host. Use -p 0.0.0.0:<port>:<container-port> for external access.

7. Docker Compose Networking

Compose creates an isolated network per project:

# Inspect the compose network
docker network ls | grep <project>
docker network inspect <project>_default

# Check DNS resolution between services
docker compose exec <service> getent hosts <other-service>

# Recreate the network
docker compose down && docker compose up -d

# Force a specific network subnet (helps with VPN conflicts)
# Add to docker-compose.yml:
# networks:
#   default:
#     ipam:
#       config:
#         - subnet: 10.100.0.0/16

8. Production Checklist

When debugging container networking in production:

# 1. Basic connectivity
docker exec <c> ping -c 2 google.com

# 2. DNS resolution
docker exec <c> nslookup google.com 8.8.8.8

# 3. Listen sockets
docker exec <c> ss -tlnp

# 4. Network config
docker exec <c> ip addr && docker exec <c> ip route

# 5. Docker-specific
docker inspect <c> --format '{{json .NetworkSettings}}' | jq .

# 6. Host-level
docker port <c>
sudo iptables -L DOCKER -n -t nat

If all else fails, remember: you can always run docker exec -it <container> bash and debug as if it's a regular Linux machine — because it is.