Packet loss

[NFL0NR]

To Kinda piggy back on the previous Packet Loss post...  check out this image..
notice the steady ~ 1% packet loss..   The drop off and reduction is only due to rebooting the MX device.  

Now I know it's not a huge deal to reboot them, but we shouldn't have to... 

[MerakiJockey505]

Couldn't agree more @NFL0NR.  It still surprises me how many troubleshooting solutions require either a reboot or a remove-from and re-add to network fix.  The majority of our sites are in production at all times and a reboot or a network pull requires a large amount of coordination with site admins and providers in order to facilitate.  Even worse, if the reboot doesn't fix the issue or ends up causing another problem we've now taken down the site for nothing and/or made it worse.  I realize this is why failover devices are important, however, some sites don't have the budget for multiple pieces of equipment. 

[BHC_RESORTS]

---

@NFL0NR wrote:

To Kinda piggy back on the previous Packet Loss post...  check out this image..
notice the steady ~ 1% packet loss..   The drop off and reduction is only due to rebooting the MX device.  

Now I know it's not a huge deal to reboot them, but we shouldn't have to... 

---

What's that WAN connection? Site to site VPN?

[NFL0NR]

Ip of the cable modem.

[BHC_RESORTS]

---

@NFL0NR wrote:

Ip of the cable modem.

---

Weird, is it doing some kind of NAT? Straight bridged mode should have the real WAN IP show up. Haven't seen that one before, except with a stupid ATT Uverse gateway that had 0 support for bridged mode - had to do a weird DMZ thing. Luckily, replaced it with a real connection.

[AlexS]

If that uplink monitor is to the internal interface of your upstream device and you're seeing packet loss then the issue can be tracked via packet capture. Generally, when you see loss on that graph it's because the MX isn't seeing those packets come back. Running a pcap on the MX would show that this is the case.  If it's not the case then that's something we'd like to look into on our side.

If you do find that a pcap shows the missing packets like you're seeing in that graph, figuring out if the issue is with the MX or the upstream device would mean running a pcap between the devices. I keep a hub on hand to put between devices while my laptop is in another port so I can run a promiscuous pcap in wireshark. If you see the packets not coming back from the upstream device, their lies the issue. If you see all the packets in this in-between capture but a simultaneous capture on the MX doesn't show them then, again, we'll want to look into this in Meraki Support. Please keep in mind that adding a hub to the mix would reset connections and you might not see loss anymore for a period of time.

From my experience, the reason you start seeing loss after a period of time that can be fixed by a reboot is because of traffic prioritization on the upstream device. When you setup a monitor like that or use the default one to 8.8.8.8, the MX sends a constant stream of ICMP to the IP address. Some devices will see this traffic as unnecessary and after a time will de-prioritize it or just drop it entirely.

During periods where you see steady 1% loss to the upstream device, do you see the same loss in the 8.8.8.8 graph?

[NFL0NR]

I hear what you're saying AlexS, but that doesn't explain why you need to reboot a router..

[AlexS]

If my theory is correct, rebooting the router resets the ICMP flow so the upstream device resets its priority. You do not have to reboot the router as there is probably no actual loss experienced by clients in times when you see it in the graph. When next you do see that 1% consistent loss, try a test from a client device behind the MX, like a PingPlotter test. You should see different results as this would be a different ICMP flow.

[MST]

I would contact support. We had similar problem and there was something on Meraki End but now that was fixed and no more pocket loss. But indeed, when we listen musing there was a choppy sound so loss indeed occurred. Since support fix that, no more issues with packet loss. 

[NFL0NR]

Alex, here's the same site's packet loss to google.. again you see steady 1% loss until the reboot..   

[AlexS]

If you are seeing this loss on client devices as well then I would agree with @MST and you should give us a call.