Cannot Connect When Port Forwarding

[Poe]

Hello,

We just got a new client who has a Meraki MX 95 (Firmware: MX 18.211.2). I am new to Meraki so thank you for your patience.

I am trying to forward a port to an internal system that is using ssh. I attempted a port forward but it is not working. 

Here is a screen grab:

Port Forward

I have tried port 22 to 22 and also with "Any" in allowed remote IPs but this made no difference. When I scan the port externally it shows as filtered. I can login to the ssh locally without any issue so I know it is not the device. 

I have attempted a Firewall log scan for port 2223 but it returns with 0 entries. 

I am not sure what else to try.

Thank you.

[RWelch]

Troubleshooting Port Forwarding and NAT Rules 

Port Forwarding and NAT Rules on the MX 

[Poe]

Thanks for your reply RWelch,

I have looked through these links and the two things I need to try are rebooting the upstream device to clear its ARP cache and seeing if the Remote IP is allowed. I will have to wait until after-hours to reboot the client's modem, but what is the best way to check if my remote IPs are being blocked?

I know the port and local IP are correct and I know there is no upstream firewall.

[RWelch]

what is the best way to check if my remote IPs are being blocked?

@PhilipDAth responded with a packet capture link below to help verify.

[alemabrahao]

Does the MX have a public IP configured directly on the WAN interface or is it behind a NAT?

[Poe]

It has a public IP on the WAN.

[alemabrahao]

Have you checked whether there are any restrictions on the System's local Firewall?

[Poe]

The Meraki has only the Default rules for all sections.

The only exception is WAN appliance services is allowing ICMP from Any and Web from none.

[PhilipDAth]

The MX config is correct.

Try a packet capture to see if you can see your SSH traffic hitting the MX.

https://documentation.meraki.com/General_Administration/Cross-Platform_Content/Packet_Capture_Overvi...

Has the device you are SSHing into got a host firewall running?  Any chance the host firewall is filtering out remote connections?

Does the host itself have its default gateway pointing to the MX?  Can it ping something like 8.8.8.8 ok?

[Poe]

I performed a quick packet capture: 

--- Start Of Stream ---
tcpdump: listening on wan0_sniff, link-type EN10MB (Ethernet), snapshot length 262144 bytes
18:52:31.053950 IP (tos 0x0, ttl 127, id 50673, offset 0, flags [DF], proto TCP (6), length 90)
    10.1.10.97.50955 > [my remote IP].443: Flags [P.], cksum 0x8c4a (correct), seq 1726003904:1726003954, ack 1446734600, win 255, length 50
18:52:31.078232 IP (tos 0x0, ttl 48, id 34554, offset 0, flags [DF], proto TCP (6), length 64)
    [my remote IP].443 > 10.1.10.97.50955: Flags [P.], cksum 0xbf88 (correct), seq 1:25, ack 50, win 501, length 24
18:52:31.135689 IP (tos 0x0, ttl 127, id 50674, offset 0, flags [DF], proto TCP (6), length 40)
    10.1.10.97.50955 > [my remote IP].443: Flags [.], cksum 0xce0e (correct), seq 50, ack 25, win 255, length 0

It is seeing my remote IP but I don't see the correct port (2223). I will check the local device. It should be stock Debian, I will check the gateway and see if ufw is present. 

---

[rhbirkelund]

You might want to ensure that the openssh-server is installed on the Debian aswell.

"apt install openssh-server"

[Poe]

sshd is running and nmap shows 22 open on the server.

[PhilipDAth]

If you jump onto the console of the Linux server and do a tcpdump - do you see the packets hitting the network interface of the server?

[Poe]

I connected to the server, it could ping 8.8.8.8 and the gateway was set to the firewall.

I could ping the gateway but DNS was not resolving google.com. I changed the gateway to openDNS (208.67.222.222) and that allowed me to resolve google.com. The firewall (ufw) is not installed and the ssh port 22 shows as open when I "nmap localhost". I am not sure of any way to check if the server is filtering remote IPs but I doubt it is.

I find it odd that the firewall gateway was not able to be used as a DNS server. Is this a clue?

[rhbirkelund]

For good measure, could you try and replace your external IP on the Forwarding rule with "Any"?

Allow the MX to update it's config, and then try to SSH to port 2223, then.

Also, you mention that the MX has a Public IP on it's WAN. Is that also what you see when browsing to Appliance Status and clicking on the Uplink tab?

[Poe]

I changed the port forward to Any. 

I checked the Application Status and the WAN1 is 74.93.111.234 the Uplink is Dynamic IP: 10.1.10.97 with gateway: 10.1.10.1

Does that mean it IS behind a NAT? Why wouldn't the WAN interface be 10.1.10.97 then?

Does this mean I have to have the ISP bridge their device or forward the port?

[rhbirkelund · Accepted solution]

Yes, if the Uplink IP is set to 10.1.10.97, that means it's being NAT'ed.

Your WAN interface simply shows your public IP address in the same way as if you'd use any tool on the internet to view your public ip, from your PC.

You'll need to put the ISP device into Bridge Mode.

[Poe]

Thank you rhbirkelund and to everyone else for your help!

[rhbirkelund]

Bridge mode, or configure your ISP router to open or forward 2223 to your MX.