All firewall rules disappeared

SOLVED
lpopejoy
A model citizen

All firewall rules disappeared

I have a network that lost all of its firewall rules sometime in the past 6 months.  Not a good situation at all as this is in healthcare...

 

The last entry in the change log regarding "firewall rules" was me - ADDING rules.  Nothing since.  I'm holding for support now, but wanted to reach out to see if anyone has seen anything like this.

 

MX84 running 14.53.

1 ACCEPTED SOLUTION
andychew
Meraki Alumni (Retired)
Meraki Alumni (Retired)

Hi Luke,

 

I want to apologize again that this issue has caused you inconvenience.

I had a chance to connect with our Support and Development Teams. The issue that you experienced could have been a result of two independent conditions:
1. With Network Objects enabled, the issue can occur when sequential steps were taken during L3 Firewall Rule configuration.
2. Unrelated changes (L7 or Forwarding Rules) on the Firewall page.

 

Our Development Team has deployed fixes for both of the conditions above. You should not expect to experience the issue again. If you do, please immediately reach out to Support. We'll be able to appropriately triage the case. We also see that you have already added back your lost firewall rules onto Dashboard.

 

Please let me know if you have any additional concerns.

 

Thank you.

View solution in original post

16 REPLIES 16
Inderdeep
Kind of a big deal
Kind of a big deal

@lpopejoy : Did you change to passthrough/NAT mode or enable 3rd party VPN? This may remove the L3 rules and the change log wouldn't record their removal.

Regards/Inder
Cisco IT Blogs awarded in 2020 & 2021
www.thenetworkdna.com

@Inderdeep No, we haven't.

Inderdeep
Kind of a big deal
Kind of a big deal

@lpopejoy : Anything changed in the dashboard ?

Regards/Inder
Cisco IT Blogs awarded in 2020 & 2021
www.thenetworkdna.com

Yes, Layer 7 FW rule was edited, and a bunch of switch ports.  

 

I opened a support ticket - he looked through everything for about an hour, but said he would have to "escalate to development".  That feels like a black hole, but we'll see.

Inderdeep
Kind of a big deal
Kind of a big deal

@lpopejoy : I saw sometimes this is an issue while changing something leads to removed firewall rules. Not sure why, Let's wait for Meraki Support to reply back with the Root Cause Analysis. 

Regards/Inder
Cisco IT Blogs awarded in 2020 & 2021
www.thenetworkdna.com

Yeah, I hope we get a good explanation.  It creates a LOT of questions for me.  (How do I know this hasn't happened somewhere else).  This could be a disaster.  Firewall rules are not items that should "just disappear".  

Inderdeep
Kind of a big deal
Kind of a big deal

@lpopejoy : yeah i agree, let's see 🙂 

Regards/Inder
Cisco IT Blogs awarded in 2020 & 2021
www.thenetworkdna.com
DarrenOC
Kind of a big deal
Kind of a big deal

Following for visibility - interested to see how this one plays out....

Darren OConnor | doconnor@resalire.co.uk
https://www.linkedin.com/in/darrenoconnor/

I'm not an employee of Cisco/Meraki. My posts are based on Meraki best practice and what has worked for me in the field.
lpopejoy
A model citizen

Ok, the answer is in, and it isn't pretty.  I'm not even sure how to respond to this.

 

RE: Cisco Meraki Case 0642****

 

Hello Luke

Thanks for the reminder, It looks like dashboards that had Network objects enabled sometimes experienced having L3 FW disappear from time to time and those changes don't appear to show up in the change log. This doesn't  happen often but you would have to manually add the L3 FW rules back to the dashboard at this point. Our development team is still looking at ways to make sure that this doesn't happen for any customer dashboards going forward in the future. In the meantime if you experience any additional issues with L3 FW rules disappearing please let me know.

Thank you, and I look forward to your response!

Je****
Cisco Meraki Technical Support

Inderdeep
Kind of a big deal
Kind of a big deal

@lpopejoy : Ok thanks for the insight, Good luck !

Regards/Inder
Cisco IT Blogs awarded in 2020 & 2021
www.thenetworkdna.com
DarrenOC
Kind of a big deal
Kind of a big deal

I must admit - that is pretty poor.

 

Because of this we're updating all our documents across all customers whom are running more complex firewall requirements.  

 

That kind of outage would ruin a reputation!

Darren OConnor | doconnor@resalire.co.uk
https://www.linkedin.com/in/darrenoconnor/

I'm not an employee of Cisco/Meraki. My posts are based on Meraki best practice and what has worked for me in the field.
DarrenOC
Kind of a big deal
Kind of a big deal

I guess the one caveat is that Network Objects are still in Beta but this really doesn't help those who are suffering outages because of it.

Darren OConnor | doconnor@resalire.co.uk
https://www.linkedin.com/in/darrenoconnor/

I'm not an employee of Cisco/Meraki. My posts are based on Meraki best practice and what has worked for me in the field.

Yes, valid point.  I thought of that too, but half of what Meraki does is in "beta".  

andychew
Meraki Alumni (Retired)
Meraki Alumni (Retired)

Hi Luke,

 

Sorry for this inconvenience.

 

I'm Andy, Technical Marketing Engineer part of the MX Product Team. I would like to further investigate this for you. I'll reach out to you via direct message. If you don't mind, please share with me this Support Case Number.

 

Thank you.

Thanks Andy, just responded!

andychew
Meraki Alumni (Retired)
Meraki Alumni (Retired)

Hi Luke,

 

I want to apologize again that this issue has caused you inconvenience.

I had a chance to connect with our Support and Development Teams. The issue that you experienced could have been a result of two independent conditions:
1. With Network Objects enabled, the issue can occur when sequential steps were taken during L3 Firewall Rule configuration.
2. Unrelated changes (L7 or Forwarding Rules) on the Firewall page.

 

Our Development Team has deployed fixes for both of the conditions above. You should not expect to experience the issue again. If you do, please immediately reach out to Support. We'll be able to appropriately triage the case. We also see that you have already added back your lost firewall rules onto Dashboard.

 

Please let me know if you have any additional concerns.

 

Thank you.

Get notified when there are additional replies to this discussion.
Welcome to the Meraki Community!
To start contributing, simply sign in with your Cisco account. If you don't yet have a Cisco account, you can sign up.
Labels