Meraki Z3 and MX - Ethernet port carrier change

[Techynerd]

I am confused by an issue with many teleworkers using Z3 and MX64 & 67 for home setups.  We have approximately 3000 active peers distributed between three MX250s.  Many teleworkers complain about lag and freezing on Citrix applications and IP phones.  Looking at the MX250s, they are running clean with no evidence of utilization issues.  This is also not happening to everyone who is peered to an MX250.

When I look at logs for the teleworker Z3 or MX appliance, I only see a lot of "Ethernet port carrier change" messages.  These usually range from every few hours to more frequently.  I know all the posts relay that this is a physical connectivity issue.  I find that hard to be accurate as we have swapped out cables for some with no change.  Also, I see this message on just about everyone's appliances, even those who do not complain of lag issues.  It just isn't possible that hundreds of users have faulty patch cables.  I do not see these messages on the MX250 hubs.

Any advice?

[RaphaelL]

I would advise to open a support case , they have a lot of visibility.

Any chance you are running into a known issue ? Are you running a old firmware ? 

[Techynerd]

RaphaelL - Thank you for the advice.  We have opened a support case with TAC.  TAC has pointed to some other things that we have taken action on and has not improved or resolved our issue.  TAC is not giving much info on the Ethernet port carrier change messages.  They have repeated the same physical connectivity line, but again this cannot be an issue with all of the thousands of appliances we have deployed and all show the same message. 

[Ryan_Miles]

What port is listed in the events? The WAN or one of the LAN ports?

[Techynerd]

Ethernet 0, the WAN port to ISP internet modem.  Every few hours there are three messages within a four seconds.  The first is "device: port0, carrier: false", followed by "device: port0, carrier: true", and then this message again "device: port0, carrier: false".

This could be a false positive or some other non-impacting message because I see it on Meraki where users are complaining and on the ones that are not having issues.  The explanation that this is a physical connectivity alert doesn't add up.

[Ryan_Miles]

Any commonality in the firmware versions of folks reporting the issues? MX16, 17, 18?

[PhilipDAth]

Has this problem always happened?

If not, can you identify when it started happening?

If so, can you identify any changes in the environment around that time?

[JoshOfTheCloth]

This seems to be an issue that is only affecting one of our sites. It's quite annoying because we have recently upgraded from the MX64 to the MX67. There was no issue with the MX64, it was just at end of life and we couldn't renew the license. The device gives 2 "Ethernet port carrier change" messages at the same time exactly. the messages are: first message "device: port0, carrier: true" second message "device: port4, carrier: true" I don't see the carrier false message and the time is always the exact same, meaning if the first message is at 'Feb 29 14:38:21' then the second is the same.

Only port 0 and 4 are in use so I originally thought it was something due to that, but after looking around a bit online, I'm not so sure. The device doesn't seem to reboot, but causes the network to drop for the customer for about 2-10 minutes. We are currently running firmware MX 18.107.2. I will try to upgrade it to MX 18.208 tonight and monitor it for the next couple of days and give an update.

[JoshOfTheCloth]

The Firmware update seems to have resolved this issue.

[JoshOfTheCloth]

The Issue was still present but went from 3-4 times a day to once a week. I reached out to Meraki and they sent a replacement device. This seems to have been a hardware issue.