MS120 - Gateway warning (bad connectivity to Dashboard, possible firewall or NAT issue)

[Trabel]

I have a Meraki MS120 that I'm trying to setup. It's on an isolated VLAN on our network. I've got an upstream firewall showing successful NAT translations and the switch reaching the cloud per the created access rules. However, when the switch reboots, it connects for 8-9 minutes then gives me the Gateway warning. Any help is appreciated.

[TheITWay]

Hello Trabel, 

If you are having the bad gateway warning, it means that the Switch is not having a reliable connection stream. 

The Meraki devices have a continuous communication to the cloud and they keep monitoring the network to ensure is reliable. They make some tests to ensure it has connectivity all the way. Some of those tests are ARPs, DNS and pings. 

I would recommend you to check which one of those is failing, most likely is related with the ARP and your gateway configuration.  The easiest way to find out is taking packet captures in the uplink of the switch and analyze the traffic to see why the connection to the gateway is failing. If you check that the connection from the switch to the cloud is good, you can take packet captures form the dashboard, otherwise would be from the upstream device. That would be easier if you have an MX upstream, you just have to take pcaps from the LAN of the MX.

I have seen these error messages when there is a static IP assignment that is not properly configured (wrong gateway or VLAN). You will be able to see that easily with pcaps as well.

You can find additional information regarding the connectivity checks here. Even though the kb mentions about the MX, the switches monitors the network the same way:
https://documentation.meraki.com/MX/Firewall_and_Traffic_Shaping/Connection_Monitoring_for_WAN_Failo...

[Trabel]

Thank you for the reply!

So I've configured vlan 333 on the up-linking router interface but it looks like the switch is setup for vlan 1. Would I need to configure the Meraki for vlan 333 as well as that is the also where it is getting it's dhcp assignment from.

[TheITWay]

Yes, the link should be configured the same. If you have VLAN 333 upstream, you should configure VLAN 333 in the switch as well. Let me know if that clears the warning message

[BlakeRichardson]

@Trabel  Have you read this knowledge base article?

https://documentation.meraki.com/zGeneral_Administration/Other_Topics/Upstream_Firewall_Rules_for_Cl...

[Trabel]

Yes Blake, I built out my firewall rules and NAT based on that article. I only however added rules for ports 80, 443, 7351, 7734, 7752... to test. I didn't think I'd need the camera proxy yet. I did add the rule for 8.8.8.8 for ICMP but it didn't change anything.

[PhilipDAth]

Is it able to get to the configured DNS servers ok?

[Trabel]

I have given it access to the Meraki DNS IP's and that didn't seem to do anything

[Trabel]

Now that I got the VLAN tagging right... I'm getting a DNS server error. I'm using active public DNS servers though in the DHCP configuration. I even see the Meraki switch going out and making DHCP calls to that IP successfully.

Any ideas?

[TheITWay]

If you are receiving DNS error message, focus completely on that. 

Take pcaps in the uplink port of the switch and follows the DNS server traffic. That could be for two main reasons:

- The switch is not receiving DNS responses from the DNS server. You should see the DNS requests form the switch and follow that traffic to understand where the problem is coming from. 

- The switch is receiving DNS responses with a different VLANs ID and it is dropping the traffic (very unlikely). 

Everything should be clear with a pcap. If you like, you can upload the pcap here and I can help you to analyze it.

[Trabel]

I see successful requests to the public DNS server I have it pointed to via event traffic in our firewall. So it's getting out fine.

Tried doing a p-cap on the interface its uplinked with and it never completes.

P-Cap results:

--- Start Of Stream ---
--- End Of Stream ---

Also getting this error now... it appears every now and then. But my DHCP server shows a successful bind.

"Bad IP assignment configuration"

[Trabel]

[TheITWay]

Regarding the pcap:
- I would recommend you to download the pcap instead of looking at it on the dashboard. 
  + Inside the pcap, take a loot at the DNS traffic and ensure it has the proper Src:Dst MAC, IP and VLAN. 

Regarding the "Bad IP assignment configuration":

- This can be addressed in the same pcap as well, verifying Src:Dst MAC, IP and VLAN of the packets
- Since the device is getting IP from DHCP, ensure you do not have any static IP assignment. If there is a discrepancy, you will have that message. The pcaps can show the discrepancy as well. 

Your best choice to find the solution is having a pcap in the upstream port and check the traffic from the device. 

99% of these problems come from configuration of the static IP assignment, and VLAN. 

You can try as well changing the DNS to google and check if the issue persists. 
  

[Trabel]

I did a Pcap with wireshark and noticed it successfully getting a DHCP offer from the server. If I haven't mentioned before, I'm using a Cisco router as a dhcp server which the Meraki is directly connected. It appears the Meraki is getting an IP offered from the router however when going to the local status page (via the management port) it says it can't get an IP. However the router and my upstream firewall says otherwise. I noticed the "Unreachable" packets and am investigating that.

[TheITWay]

Good that you have the pcap, I notice the following:

- There is no DHCP request coming from the Meraki device
  + If it is not in the pcap, could be that the Meraki device does not like the offer or the packet itself, check the MAC and VLAN info in that packet

- The Meraki device is not getting ping replies. Therefore, it will flag an error message such as "Bap IP assignment" or "Bad connectivity". 

- The Meraki device is sending DNS request but there is no DNS replies. Therefore, you will see the "DNS misconfigured"

The behavior you are having is not just related on getting an IP address, you should ensure all the traffic coming from the device, the upstream device is able to forward the reply. 

Everything points out you have an issue upstream. Let me know what you find in your investigation. 

[Trabel]

Thank you for the great info. I managed to get the switch statically IP’d and am able to see it now talking on the network. 

Upstream the packets appear to be getting to their destined addresses. The IP the meraki has is hitting the public DNS servers I’ve given it, though I don’t see any return traffic coming back in the firewall from those DNS servers. Even though meraki’s site says it only needs outbound rules I put an inbound allow for return traffic. 

Basic IP Connectivity on my network shows the meraki can traverse back and forth just fine. Now that it has a static IP the Connection status page still says switch is trying to join the network. 

[Trabel]

I found the solution.

Our network was able to send the private Meraki network out through our upstream firewall but our firewall didn't have a route for when the traffic came back in. Once I added that route, it worked.

[TheITWay]

Great!

I am glad that you were able to find the problem.

Everything was pointing out to an upstream issue :). The next time you have this kind of behavior, you can use my last point as reference for troubleshooting.

I am glad that you cleared this out. Until the next one then.