Getting error connection reset when loading a web page after Firewall changeover

[JR_216]

Hello all,

I am a K12 Sysadmin who is new to the Meraki landscape. We are currently in the midst of an E-Rate changeover from our older networking equipment (Fortigate firewalls and HP Aruba Switches).

We recently pulled our Fortigate firewalls from all sites and configured an MX250 at our main MDF and MX105 at our sub sites. everything is configured and passing traffic but there is some odd behaviors happening.

Since changing the firewalls roughly every 20 minutes or so whenever a webpage is clicked it doesn't load and eventually times out with an error page saying "Error connection reset" and the reload button below that. After clicking the reload page button the page loads fine and we are on our way.

Originally this was just a minor inconvenience as I assumed this was just some initial bugs that we would out. We are a month into the install now and are adding switches currently and the issue seems the same as before.

I have went through the firewall and whitelisted as many of the main sites the district uses. I have turned off IPS/IDS, AMP is still on currently, and am currently on firmware 18.107.7.

I don't have a ton of hair left but am currently pulling it out trying to understand what is going on. I had none of these issues when I was using my Fortigate Firewalls. Please send help. My patchy skull needs it.

Edit:  I also seem to be getting this error every 2 minutes on the MX250

[alemabrahao]

If you could provide the network topology it would be of great help in trying to understand your scenario.

If you think everything is ok, I recommend opening a support case.

[JR_216]

L3L2

Attached is a SS of my L3 and L2 Topology from the dashboard.  If you need it more drawn out or explained I can provide that as well

[alemabrahao]

I'm curious to know why there are two paths to that MX105?

[JR_216]

Must be a mistake from the company that was Setting up the VLANS.  Zero clients route through that route so I am unsure why they would have put that in.  They are here on Wednesday for a few days for more installs so I will have a chance to ask them then.  But I agree that it doesn't make sense to be there.  

[RaphaelL]

Hi there ,

Error connection reset  usualy mean that you received a TCP reset. One thing that I like to know is who sent the tcp reset ? The end server or some device in the middle. 

To do so , take a packet capture and try to capture the TCP reset. Observe in the IP header the TTL. Is the TTL the same for other packets from the server ?

Eg :  

Packets 1-2-3-4-5-6 have a TTL of 114

TCP reset packet 7 has a TTL of 64.

This is just a tip to try to find who is sending the reset.

[ww]

Is Client tracking on the mx set to mac?

https://documentation.meraki.com/MX/Monitoring_and_Reporting/Client-Tracking_Options

I have read posts where things like this where fixed by using IP or UCI

[JR_216]

Currently Yes they are set as Identified by MAC address.  IP is unselectable and UCI is in Beta but is claimed to be recommended for the network

[ww]

Maybe you could try UCI  for 1 spoke network

IP can only be used on non combined network. Then you would need to split the network to get a single mx network.

[JR_216]

I am a little nervous to change it as it suggests a bunch of VLAN changes that I am not comfortable dropping for just turning that on

[mlefebvre]

By any chance do you have Bonjour Forwarding configured in any of those sites?

[JR_216]

I do not.

[mlefebvre]

Then I would get a case open with Meraki support asap, get on the phone with them and have them examine the MX logs while it is happening, they have access to significantly more information than what is presented through the dashboard.

[ww]

Regarding the mx250. Thats not good. What is the utilization of that device? 

Also , get in touch with meraki support

https://documentation.meraki.com/MX/Monitoring_and_Reporting/Device_Utilization#MX_Device_Utilizatio...

[JR_216]

Assuming I am in the right place to check this.  Organization----summary report----top security appliances by utilization is only 5%

[BlakeRichardson]

That report only shows traffic. What is the CPU and throughput utilisation of the MX?  You might have the ask support to get this information.

Also where are the devices connected to the MX105 getting their DNS from, could slow DNS be causing the web browser to timeout. 

[JR_216]

I wanted to give an update on this.  Support couldn't isolate the issue because it was user wide and random so packet captures were near impossible.  Turns out that putting on any Content Filtering with Category Blocking is causing all websites to time out or fail to load.  I removed all Content and Threat categories from the list and suddenly zero sites have an issue.  The minute I add anything back to either of those lists it bogs down and nothing will load without timing out. 

I am unsure if this is firmware related or not but I am at 18.107.7

[The_DONALD]

Yes, seeing same issue.  Confirmed from Meraki support that our MX is "being affected by the current unexpected behavior with Content filtering and website blocking".  They're saying that the engineering team is working on a fix, but we're a few days deep into this and not looking good.  Any luck with a workaround or anything?  I will need to look at deploying a new solution if this goes much longer

[JR_216]

The Current theory is it stems from tracking clients via their MAC Address and not by using the UCI Tracking. I won’t be able to change it to the UCI tracking till later this week to test the change. The other option would be a roll back all the way to 16.6 and setting the filtering to top list.