All firewall rules disappeared

SOLVED
lpopejoy
Building a reputation

All firewall rules disappeared

I have a network that lost all of its firewall rules sometime in the past 6 months.  Not a good situation at all as this is in healthcare...

 

The last entry in the change log regarding "firewall rules" was me - ADDING rules.  Nothing since.  I'm holding for support now, but wanted to reach out to see if anyone has seen anything like this.

 

MX84 running 14.53.

1 ACCEPTED SOLUTION

Hi Luke,

 

I want to apologize again that this issue has caused you inconvenience.

I had a chance to connect with our Support and Development Teams. The issue that you experienced could have been a result of two independent conditions:
1. With Network Objects enabled, the issue can occur when sequential steps were taken during L3 Firewall Rule configuration.
2. Unrelated changes (L7 or Forwarding Rules) on the Firewall page.

 

Our Development Team has deployed fixes for both of the conditions above. You should not expect to experience the issue again. If you do, please immediately reach out to Support. We'll be able to appropriately triage the case. We also see that you have already added back your lost firewall rules onto Dashboard.

 

Please let me know if you have any additional concerns.

 

Thank you.

View solution in original post

16 REPLIES 16
Inderdeep
Kind of a big deal

@lpopejoy : Did you change to passthrough/NAT mode or enable 3rd party VPN? This may remove the L3 rules and the change log wouldn't record their removal.

Regards
Inderdeep Singh
www.thenetworkdna.com ( Awarded by Cisco IT Blogs award 2020)
lpopejoy
Building a reputation

@Inderdeep No, we haven't.

Inderdeep
Kind of a big deal

@lpopejoy : Anything changed in the dashboard ?

Regards
Inderdeep Singh
www.thenetworkdna.com ( Awarded by Cisco IT Blogs award 2020)
lpopejoy
Building a reputation

Yes, Layer 7 FW rule was edited, and a bunch of switch ports.  

 

I opened a support ticket - he looked through everything for about an hour, but said he would have to "escalate to development".  That feels like a black hole, but we'll see.

Inderdeep
Kind of a big deal

@lpopejoy : I saw sometimes this is an issue while changing something leads to removed firewall rules. Not sure why, Let's wait for Meraki Support to reply back with the Root Cause Analysis. 

Regards
Inderdeep Singh
www.thenetworkdna.com ( Awarded by Cisco IT Blogs award 2020)
lpopejoy
Building a reputation

Yeah, I hope we get a good explanation.  It creates a LOT of questions for me.  (How do I know this hasn't happened somewhere else).  This could be a disaster.  Firewall rules are not items that should "just disappear".  

Inderdeep
Kind of a big deal

@lpopejoy : yeah i agree, let's see 🙂 

Regards
Inderdeep Singh
www.thenetworkdna.com ( Awarded by Cisco IT Blogs award 2020)
UCcert
Kind of a big deal

Following for visibility - interested to see how this one plays out....

Darren O'Connor | uccert.co.uk
https://www.linkedin.com/in/darrenoconnor/

I'm not an employee of Cisco/Meraki. My posts are based on Meraki best practice and what has worked for me in the field.
lpopejoy
Building a reputation

Ok, the answer is in, and it isn't pretty.  I'm not even sure how to respond to this.

 

RE: Cisco Meraki Case 0642****

 

Hello Luke

Thanks for the reminder, It looks like dashboards that had Network objects enabled sometimes experienced having L3 FW disappear from time to time and those changes don't appear to show up in the change log. This doesn't  happen often but you would have to manually add the L3 FW rules back to the dashboard at this point. Our development team is still looking at ways to make sure that this doesn't happen for any customer dashboards going forward in the future. In the meantime if you experience any additional issues with L3 FW rules disappearing please let me know.

Thank you, and I look forward to your response!

Je****
Cisco Meraki Technical Support

Inderdeep
Kind of a big deal

@lpopejoy : Ok thanks for the insight, Good luck !

Regards
Inderdeep Singh
www.thenetworkdna.com ( Awarded by Cisco IT Blogs award 2020)
UCcert
Kind of a big deal

I must admit - that is pretty poor.

 

Because of this we're updating all our documents across all customers whom are running more complex firewall requirements.  

 

That kind of outage would ruin a reputation!

Darren O'Connor | uccert.co.uk
https://www.linkedin.com/in/darrenoconnor/

I'm not an employee of Cisco/Meraki. My posts are based on Meraki best practice and what has worked for me in the field.
UCcert
Kind of a big deal

I guess the one caveat is that Network Objects are still in Beta but this really doesn't help those who are suffering outages because of it.

Darren O'Connor | uccert.co.uk
https://www.linkedin.com/in/darrenoconnor/

I'm not an employee of Cisco/Meraki. My posts are based on Meraki best practice and what has worked for me in the field.
lpopejoy
Building a reputation

Yes, valid point.  I thought of that too, but half of what Meraki does is in "beta".  

Hi Luke,

 

Sorry for this inconvenience.

 

I'm Andy, Technical Marketing Engineer part of the MX Product Team. I would like to further investigate this for you. I'll reach out to you via direct message. If you don't mind, please share with me this Support Case Number.

 

Thank you.

lpopejoy
Building a reputation

Thanks Andy, just responded!

Hi Luke,

 

I want to apologize again that this issue has caused you inconvenience.

I had a chance to connect with our Support and Development Teams. The issue that you experienced could have been a result of two independent conditions:
1. With Network Objects enabled, the issue can occur when sequential steps were taken during L3 Firewall Rule configuration.
2. Unrelated changes (L7 or Forwarding Rules) on the Firewall page.

 

Our Development Team has deployed fixes for both of the conditions above. You should not expect to experience the issue again. If you do, please immediately reach out to Support. We'll be able to appropriately triage the case. We also see that you have already added back your lost firewall rules onto Dashboard.

 

Please let me know if you have any additional concerns.

 

Thank you.

View solution in original post

Get notified when there are additional replies to this discussion.
Welcome to the Meraki Community!
To start contributing, simply sign in with your Cisco account. If you don't yet have a Cisco account, you can sign up.
Labels