Upgrade firmware on a network with two switchs loses cloud connection and gets 1.1.1.1 address

CharlieCrackle
Getting noticed

Upgrade firmware on a network with two switchs loses cloud connection and gets 1.1.1.1 address

 

 

I upgraded a network with 2 x MS120 switches to  12.28 (stable)   and then lost the switches and connection to cloud.

 

Just before having to drive 80km to power cycle them  the MX started complaining about Duplicate IP 1.1.1.1  and the two mac addresses were the switches.

 

Why does the meraki use a public ip of 1.1.1.1   ?

 

Why do the switches boot with this ip ?  (I have them set for DHCP)

 

I did a wireshark capture and both switches are arping 8.8.8.8   and getting no where.....

 

can some one explain what is going on here    makes no sence..

 

 

wiresharkwireshark

 

 

15 REPLIES 15
PhilipDAth
Kind of a big deal

Re: Upgrade firmware on a network with two switchs loses cloud connection and gets 1.1.1.1 address

I've never seen a switch default to using 1.1.1.1.  Are they statically configured?  Perhaps double check their IP configuration.

 

ps.  The switches should auto roll back the upgrade if they can't communicate with the Meraki cloud.  So try waiting a bit longer (30 minutes) before jumping into the car.

BlakeRichardson
Kind of a big deal

Re: Upgrade firmware on a network with two switchs loses cloud connection and gets 1.1.1.1 address

1.1.1.1 isn't Meraki thats Cloudflares DNS.

 

What are your MX DNS settings?

Meraki CMNO, Ruckus WISE, Sonicwall CSSA, Allied Telesis CASE & CAI
CharlieCrackle
Getting noticed

Re: Upgrade firmware on a network with two switchs loses cloud connection and gets 1.1.1.1 address

no both switches are set to DHCP.   it has been 9 hours now and they are still doing this.  what is wierd is all the cameras and AP that are connected to the switch are showing as not connected to cloud also but are working ok.  and the users on the switch are still working ok.  I don't get it at all and at this point support don't either.  Some one is going to site to power cycle the switches.

CharlieCrackle
Getting noticed

Re: Upgrade firmware on a network with two switchs loses cloud connection and gets 1.1.1.1 address

the MX is set to use Cisco umbrella DNS.   but the arp ing shows 1.1.1.1  (mac of switch)  trying to arp for the address of 8.8.8.8.

 

Had this happen to another single switch site now.   this site the switch is still working  but not connected to cloud  it was interesting as a power cycle did not fix the issue but the users were not effected and switch was still passing traffic

8-12 hours later the switch started talking to cloud its self with the last error being "Disabled Switch Bad DNS"

 

 

I am wondering if this is a MX DHCP issue  as all these issue started when the MXs were upgraded to 15.42  from 14  to fix the "Unable to download configuration error" which was happening more frequently.

CharlieCrackle
Getting noticed

Re: Upgrade firmware on a network with two switchs loses cloud connection and gets 1.1.1.1 address

Snap156.pngDoing a wire shark capture using the MAC address of the switch  you can see it making DHCP requests on native vlan and other vlan and getting responses  !

 

CharlieCrackle
Getting noticed

Re: Upgrade firmware on a network with two switchs loses cloud connection and gets 1.1.1.1 address

I just upgraded my office switches to 14.16  and got a duplicate IP 1.1.1.1  alert  when doing it.   do the switches boot with address of 1.1.1.1 ???   something wierd is going on here.  anyone else getting 1.1.1.1 duplicate ip alerts ??Snap.jpg

cmr
Kind of a big deal
Kind of a big deal

Re: Upgrade firmware on a network with two switchs loses cloud connection and gets 1.1.1.1 address

I upgraded a stack of three MS210s to 14.16 three weeks ago and didn't have any issues.  Their management interfaces are set using DHCP, are you using static IPs for management?

cmr
Kind of a big deal
Kind of a big deal

Re: Upgrade firmware on a network with two switchs loses cloud connection and gets 1.1.1.1 address

I just upgraded another site with MS120s, 210s, 220s and 225s from 14.12 to 14.16.  No issues like you experienced @CharlieCrackle 

CharlieCrackle
Getting noticed

Re: Upgrade firmware on a network with two switchs loses cloud connection and gets 1.1.1.1 address

No all set to DHCP

DHAnderson
A model citizen

Re: Upgrade firmware on a network with two switchs loses cloud connection and gets 1.1.1.1 address

I have upgraded MS120from 14.12 to 14.16 without me any issues.

 

The switches are set to get their IP addresses form DHCP on a management VLAN.

 

Are you switches on a the default VLAN, or one you specified?

cmr
Kind of a big deal
Kind of a big deal

Re: Upgrade firmware on a network with two switchs loses cloud connection and gets 1.1.1.1 address

@DHAnderson I've upgraded switches with static and DHCP management interface IPs, some on default VLAN and some not (the ones that are not were all on DHCP).  But none were MS125s, I wonder if the issue applies to them?

DHAnderson
A model citizen

Re: Upgrade firmware on a network with two switchs loses cloud connection and gets 1.1.1.1 address

@cmr ,

 

@CharlieCrackle is having issues with MS120, not MS125

 

I have lost a MS when the management VLAN set in the switch wasn't on the uplink port's allowed VLAN list. The switch was working, it just couldn't get to the dashboard.

 

I wonder if the isn't something like that.

 

 

cmr
Kind of a big deal
Kind of a big deal

Re: Upgrade firmware on a network with two switchs loses cloud connection and gets 1.1.1.1 address

@DHAnderson I think he has MS125s, look at the screenshot of them with the red arrows on.

CharlieCrackle
Getting noticed

Re: Upgrade firmware on a network with two switchs loses cloud connection and gets 1.1.1.1 address

Yes issue seen on 120s  and 125s  and switches on DHCP and default VLAN 1

DHAnderson
A model citizen

Re: Upgrade firmware on a network with two switchs loses cloud connection and gets 1.1.1.1 address

@CharlieCrackle

 

The uplink port is set to allow VLAN 1 or all VLANS?

Welcome to the Meraki Community!
To start contributing, simply sign in with your Cisco account. If you don't yet have a Cisco account, you can sign up.