ALERT: Cisco 9000 switch port channel config being erased in monitor mode

[cmr]

Please be aware that we and at least one other company have had all port channel configs defaulted on Cisco Catalyst 9000 switches that are connected in monitor mode to the Meraki dashboard.  See discussion here: Re: Port-Channels were Defaulted during the night - The Meraki Community

In our case it was a stacked pair of 9300s last night at 1am UTC and all six port channels had their configuration defaulted at the same time.  Nothing else changed at all.

I have logged a support ticket asking for urgent developer involvement.

[Brash]

Jeez that's messy! Interesting to know what caused it for everyone to be impacted at the same time.

I had expected something like that might happen in management mode but the fact it occured in monitor mode is deeply concerning.

[ConnorL]

Hi @cmr @Brash ,

Please follow this thread for the latest information about this issue: https://community.meraki.com/t5/Meraki-Service-Notices/Cloud-Monitored-Catalyst-switches-issue/ba-p/...

Kind regards,

Connor.

[cmr]

@ConnorL why after 8 hours of being acknowledged is there no update?  This took down one of our venues for several hours while we tried to figure out what had gone wrong, not what we expect from Cisco.

[Brash]

The thread has been updated to outline that engineering identified the impacted devices and Meraki is reaching out to customers.

What it's lacking however is any detail around why it occured, and whether further safeguards are being put in place to prevent anything like this from occurring in future.

I would expect monitor mode to be completely hands-off in terms of the ability to change configuration on a switch.

[cmr]

We didn't get a banner, but had already reported, so no need I guess.  However as we set the Meraki user privilege to 2 we now have a message saying that the switch config is out of date, so that at least means something has been tried.

[cmr]

Or perhaps the dashboard tried to delete the port channels again... 😱

[cmr]

Our ticket was also not updated so I've chased, as we're no further forward in knowing what the next steps are supposed to be 😡