Secure/VPN not working for SOME internet providers

[PineTree]

Hi,

We use port 446 for our Secure Client port, SAML to Azure. It works reliably with Spectrum/Charter users, and Verizon and Firstnet cellphone hotspot users.

However we have some people with other local fiber internet providers who have issues connecting. It will timeout. When I have them connect to a backup site on port 443, it seems they can connect fine. Also, these people can ping our dynamic-m URL and our public IP no problem.

Ive gone round and round with the fiber ISP support about this and they say definitively they do not block port 446, and demonstrate they can ping our IP, as well as telnet to port 446. Only variable here seems to be the port. The only reason we dont use 443 is that Secure Client/Anyconnect did not work on AT&T hotspots for us on that port.

[RWelch]

Possibly related to the issues mentioned/described above?

Cisco Cloud Security Service Status 

[PineTree]

No, I dont think so, ive been working on this for a few months.

[RWelch]

What do your packet captures and client logs indicate?  I would think they would provide good insight in diagnosing handshake failures.

[PineTree]

Connections are reaching the MX and showing fragmentation on the WAN packet capture. I just dont understand why packets might fragment and fail to connect Secure Client on port 446 but not 443 or why this might prevent connecting from a regional ISP when a national ISP like Verizon would not have an issue.

---

18:38:13.085276 IP (tos 0x0, ttl 64, id 42445, offset 0, flags [+], proto TCP (6), length 1428)
xx.xx.xxx.xx.446 > xx.xxx.xxx.xxx.18019: Flags [.], seq 2309454382:2309455770, ack 601705921, win 251, length 1388
18:38:13.085276 IP (tos 0x0, ttl 64, id 42445, offset 1408, flags [none], proto TCP (6), length 92)
xx.xx.xxx.xx > xx.xxx.xxx.xxx: ip-proto-6
18:38:17.005254 IP (tos 0x0, ttl 64, id 42446, offset 0, flags [+], proto TCP (6), length 1428)
xx.xx.xxx.xx.446 > xx.xxx.xxx.xxx.18019: Flags [.], seq 0:1388, ack 1, win 251, length 1388
18:38:17.005254 IP (tos 0x0, ttl 64, id 42446, offset 1408, flags [none], proto TCP (6), length 92)
xx.xx.xxx.xx > xx.xxx.xxx.xxx: ip-proto-6
18:38:24.775289 IP (tos 0x0, ttl 64, id 42447, offset 0, flags [+], proto TCP (6), length 1428)
xx.xx.xxx.xx.446 > xx.xxx.xxx.xxx.18019: Flags [.], seq 0:1388, ack 1, win 251, length 1388
18:38:24.775289 IP (tos 0x0, ttl 64, id 42447, offset 1408, flags [none], proto TCP (6), length 92)
xx.xx.xxx.xx > xx.xxx.xxx.xxx: ip-proto-6
18:38:31.165276 IP (tos 0x0, ttl 64, id 58802, offset 0, flags [+], proto TCP (6), length 1428)
xx.xx.xxx.xx.446 > xx.xxx.xxx.xxx.18007: Flags [.], seq 155509429:155510817, ack 2096241983, win 251, length 1388

 Im wondering if Anyconnect has separate MTU settings from the firewall. Our MTU is set to 1432 on the firewall via support.

When I connect to the VPN and test with ping -f -l I get a largest success of 1362. Does it make sense then that the meraki secure client MTU should be set to 1362 + 28 = 1390

[PineTree]

So I did a few packet captures and what I noticed is that the people on the regional Fiber ISPs had packet fragmentation issues whereas people on Charter/Spectrum and Verizon/ATT did not have fragmentation issues. I tested further and manually set the MTU on the people having issues to 1390 after some ping tests and that resolved the connection issues. 

[PhilipDAth]

Is your MX behind a device that does NAT?

If so, are you allowing BOTH TCP and UDP 446?