All firewall rules disappeared

[lpopejoy]

I have a network that lost all of its firewall rules sometime in the past 6 months.  Not a good situation at all as this is in healthcare...

The last entry in the change log regarding "firewall rules" was me - ADDING rules.  Nothing since.  I'm holding for support now, but wanted to reach out to see if anyone has seen anything like this.

MX84 running 14.53.

[Inderdeep]

@lpopejoy : Did you change to passthrough/NAT mode or enable 3rd party VPN? This may remove the L3 rules and the change log wouldn't record their removal.

[lpopejoy]

@Inderdeep No, we haven't.

[Inderdeep]

@lpopejoy : Anything changed in the dashboard ?

[lpopejoy]

Yes, Layer 7 FW rule was edited, and a bunch of switch ports.  

I opened a support ticket - he looked through everything for about an hour, but said he would have to "escalate to development".  That feels like a black hole, but we'll see.

[Inderdeep]

@lpopejoy : I saw sometimes this is an issue while changing something leads to removed firewall rules. Not sure why, Let's wait for Meraki Support to reply back with the Root Cause Analysis. 

[lpopejoy]

Yeah, I hope we get a good explanation.  It creates a LOT of questions for me.  (How do I know this hasn't happened somewhere else).  This could be a disaster.  Firewall rules are not items that should "just disappear".  

[Inderdeep]

@lpopejoy : yeah i agree, let's see 🙂 

[DarrenOC]

Following for visibility - interested to see how this one plays out....

[lpopejoy]

Ok, the answer is in, and it isn't pretty.  I'm not even sure how to respond to this.

RE: Cisco Meraki Case 0642****

Hello Luke

Thanks for the reminder, It looks like dashboards that had Network objects enabled sometimes experienced having L3 FW disappear from time to time and those changes don't appear to show up in the change log. This doesn't  happen often but you would have to manually add the L3 FW rules back to the dashboard at this point. Our development team is still looking at ways to make sure that this doesn't happen for any customer dashboards going forward in the future. In the meantime if you experience any additional issues with L3 FW rules disappearing please let me know.

Thank you, and I look forward to your response!

Je****
Cisco Meraki Technical Support

[Inderdeep]

@lpopejoy : Ok thanks for the insight, Good luck !

[DarrenOC]

I must admit - that is pretty poor.

Because of this we're updating all our documents across all customers whom are running more complex firewall requirements.  

That kind of outage would ruin a reputation!

[DarrenOC]

I guess the one caveat is that Network Objects are still in Beta but this really doesn't help those who are suffering outages because of it.

[lpopejoy]

Yes, valid point.  I thought of that too, but half of what Meraki does is in "beta".  

[andychew]

Hi Luke,

Sorry for this inconvenience.

I'm Andy, Technical Marketing Engineer part of the MX Product Team. I would like to further investigate this for you. I'll reach out to you via direct message. If you don't mind, please share with me this Support Case Number.

Thank you.

[lpopejoy]

Thanks Andy, just responded!

[andychew · Accepted solution]

Hi Luke,

I want to apologize again that this issue has caused you inconvenience.

I had a chance to connect with our Support and Development Teams. The issue that you experienced could have been a result of two independent conditions:
1. With Network Objects enabled, the issue can occur when sequential steps were taken during L3 Firewall Rule configuration.
2. Unrelated changes (L7 or Forwarding Rules) on the Firewall page.

Our Development Team has deployed fixes for both of the conditions above. You should not expect to experience the issue again. If you do, please immediately reach out to Support. We'll be able to appropriately triage the case. We also see that you have already added back your lost firewall rules onto Dashboard.

Please let me know if you have any additional concerns.

Thank you.