Bidirectional connectivity issue between Client VPN and Docker network

[AdielMC]

Hello all! 

I have a Docker network (10.251.102.0/25) on my LAN (10.251.101.0/24). Clients can connect to the Meraki VPN (Client VPN) without issues. VPN clients can ping and SSH into devices on the LAN, but they cannot access the Docker network.

While LAN clients can ping and access devices on the Docker network, devices on the ClientVPN network cannot ping or access the Docker network. The Docker containers can ping and SSH into services on the ClientVPN, but the reverse (ClientVPN to Docker) is not working.

Key Points:

• Docker Network: 10.251.102.0/25
• LAN Network: 10.251.101.0/24
• VPN Network: 10.251.200.0/24
• Docker Host Interfaces:
  • enx3c18a0d4bcae (LAN interface)
  • br-5fd12a88e33c (Docker bridge)

Routing Configuration on Docker Host:

• default via 10.251.101.1 dev enx3c18a0d4bcae proto dhcp metric 100
• 10.251.101.0/24 dev enx3c18a0d4bcae proto kernel scope link src 10.251.101.32 metric 100
• 10.251.102.0/25 dev br-5fd12a88e33c proto kernel scope link src 10.251.102.1
• 10.251.200.0/24 via 10.251.101.1 dev enx3c18a0d4bcae\

Static route on Meraki:

Stat Version Subnet Name VLAN Next hop Destination Type

	4	10.251.102.0/25	Test-docker	—	10.251.101.32	—	Local Static Route
	4	10.251.200.0/24	—	—	Client VPN	—	Client VPN

Troubleshooting Steps Taken:

1. No iptables Rules Blocking Traffic: Confirmed there are no iptables rules blocking traffic on the Docker host.

2. TCPdump on Docker Host: Captured traffic on the Docker host interface, observing ICMP request and reply traffic between the ClientVPN and Docker.

3. Wireshark Capture on ClientVPN Interface (Meraki): Captured traffic on the Meraki interface and observed ICMP requests from the ClientVPN but no responses sent back to the ClientVPN.

4. Traceroute from ClientVPN to Docker: Performed a traceroute to 10.251.102.2 (Docker network IP) and the results were as follows:

   traceroute to 10.251.102.2 (10.251.102.2), 30 hops max, 60 byte packets
   1 * * *
   2 10.251.101.1 (10.251.101.1) 912.677 ms 918.129 ms 926.002 ms
   3 10.251.101.32 (10.251.101.32) 936.121 ms 975.331 ms 975.249 ms
   4 * * *
   5 * * *
   6 * * *
   7 * * *

Key Observations:

• The ClientVPN can reach the Docker host, but there seems to be no response from the Docker network to the ClientVPN.
• The traceroute from the ClientVPN suggests that the packets are being routed correctly up to the Docker host interface (10.251.101.32), but they do not continue to the Docker network.

I would really appreciate any insights or suggestions to help resolve this issue.

Thank you in advance for your help!

[Mloraditch]

Have you packet captured the switch port connected to the physical host where the docker is? You need to see if the packets are actually leaving the host, if they are, then you can probably easily call Meraki support and find where the issue is. Your topology seems sound, but based on the one way traceroute it seems the hangup may be in the docker host. If that's the case you may need to get some help from a linux resource. Although who knows what our fellow members might know! 🙂