Loosing connectivity in the network after creating a new VLAN

[KarlDorme]

Hello,

Last month I started having issues when I create a new vlan in a network (issue is not solved yet, but case has been opened) and this morning I have a second network in our organization where I observe the same.

After the creation of the vlan, we lost connectivity to devices in other vlan's while the gateway ip's remained reachable.
f.e. 

I create a new vlan 6, ip 10.44.30.96/27 gateway-ip 10.44.30.126

I could not ping the gateway ip from the dashboard (using tools) which is already strange.

Suddenly I start loosing clients, in other vlan's, I observed switches in other vlan's which became unreachable etc...

Deleting the vlan solved the issue, all clients (pc/printers) are reachable again, error's disappear for wireless access points and switches in the portal.

I have this now already with two different networks in our environment and, on, different hardware platforms.

On one network our layer3 is running on MS210-48FP
On the other network our layer3 is running on MS410-16

Both are on MS 14.33.1

I have not tried it on a 3th network yet ;-)...

Anyone else observed the same ?

[Bruce]

What are the IP subnets on the other VLANs you have? I’m wondering if you are accidentally overlapping VLANs. Normally the Meraki Dashboard stops this or warns about it, but maybe somehow it’s not detecting the overlap.

[KarlDorme]

100% sure , even 1000% sure there is no overlapping

[KarlDorme]

So, for this morning, removing the vlan was not enough.
Some devices in different vlan's were reachable, other not ... very random.

Forget client did not solved the issue (since clear arp does not exist).
I rebooted bot core devices now, because I did not found any workaround.
All up again ...

[KarstenI]

This sounds exactly like the known bug that L3 switches stop L3 forwarding after changes in the L3 config. L2 forwarding typically is not affected. I was running into this bug many times. What helps:

• If the problem is already there, reload of the switch.
• When the switch gets reloaded before the change I didn't see this problem.
• I have the impression that it happens more often it the switch was not reloaded another time after an update.   I now always do an additional reload and it happens not that often any more.

[MarcP]

---

@KarstenI wrote:

•   I now always do an additional reload and it happens not that often any more.

---

So it still happens ? :S

Sounds like Meraki needs to know about it more so they fix it within an upcoming firmwareupdate .

[KarstenI]

The problem was discussed before. It seems they are not able to really reproduce the problem.

I think the uptime of the switch is part of the problem. That a long uptime with no extra reload is more problematic than a longer uptime with a reload.

[KarlDorme]

I was able to reproduce it on a switch which was restarted , now waiting for a webex with Meraki to do the same thing, out of office hours for our affiliate ..

They NEVER heard about this issue, I was told 😉 ... would have been strange ... isn't it ???

They wanted to see it with their own eyes, even though we provided wireshark captures, the support files from the switches etc, during an outage...

[KarlDorme]

can you provide me a link to the previous discussion or a bug number ?

[KarstenI]

Not sure if I find it any more. It was also some times on the "Known Issues" list of the release notes.

[cmr]

14.32 was the officially affected version, perhaps it persisted to a later version for you:

• In rare scenarios, changes made to SVIs may result in connectivity loss for one or more SVIs until reboot (present since MS 14.31)

It definitely existed in 14.32 for MS355X switches in a stack as we experienced the bug, but were lucky in that reversing the change restored connectivity until we could upgrade.  We haven't seen the bug since, but other experience may differ.

[KarstenI]

Lucky you ... 😉