High CRC Error's Across Multiple MS-120-48 ports

[MatthewH]

Anybody have any experience troubleshooting CRC error issues across multiple switch ports? Seems to vary from day-to-day what ports are having the issues and it's across multiple OS and devices including an MR33, windows server 2012, WIN7 pc's, and more. Ran cable tests with no issues, I find it hard to believe that this many devices are failing to auto-negotiate correct speed/duplex settings or have failing NIC's. I also wouldn't think that this many devices would have corrupted files or errors causing it. Any help is appreciated. 

[cta102]

Once had something like that with badly installed structured cabling.

Is there any commonality with the ports, can a couple of the cables in question be moved onto another switch to see if the problem moves with them?

[MatthewH]

There's close to 20 ports with a range of CRC error's, some being constant, others are intermittent. No commonalities except they all go to same patch panel, same switch. I find it hard to believe that a wiring issue would be causing this as it was all tested once wiring was done. That said next time I'm onsite I will try swapping some cables and running cable tests. Thanks for the ideas

[AjitKumar]

Hi

What are your comments  on Firmware? Hope you are running the latest stable Version.

[MatthewH]

I verified all firmware is up-to-date with stable releases on MX, MR, & MS.

[cta102]

If it was a regular Cisco switch and they were all ports serviced by the ASIC (usually groups of 4 or 8 ports) it would suggest an issue, but I haven't seen any info regarding the actual hardware of Meraki switches and the faults we have seen tend to be really obvious, or software issues (so far.)

Has the issue just started without warning, has a small issue got worse, has anything changed or is it a new install?

It may not hurt to give the Meraki helpdesk guys a call and ask them if there are any known issues.

[MatthewH]

I opened ticket with meraki support as you suggested just to check for any known issues. Can't rule out cabling just yet but I'll try to keep updated as far as what the resolution is for this going forward.

[BrandonS]

I would consider getting a real cable tester on some of those cables (like a Fluke).  CRC errors are almost always cable issue in my experience and the Meraki cable test is only very basic and would not show electrical interference issues I don't think.  Is there any heavy machinery or equipment that could interfere?  Good luck and let us know what you find out.

[Adam]

You could try moving some of the cables to ports in a different group (I think they are in groups of 8/16).  You also may want to try a spare switch if you have the option to.  Wouldn't hurt to eliminate variables.  

[PhilipDAth]

@Adam the ASCIS that Meraki use have 24 ports on them.  So a 24 port has a single ASIC, and a 48 port has two ASICS.

[BlakeRichardson]

As a means of testing whether its a cable fault can you simply connect one of the devices on an affected port using a long patch lead instead of your exsiting cabling and see if the fault goes away?

[MatthewH]

Thanks for the input gents, I have to find time to go onsite to test cabling with our tester. I don't forsee cabling on 20 different ports being the issue but just to eliminate variables I understand the need for testing. If that all tests fine I will try moving these devices onto ports 1-24 on switch to eliminate a hardware failure on the 2nd ASIC as all on 1-24 appear fine. 

[cta102]

Philip,

I don't suppose you know of any doc that lists the ASIC layout for Meraki switches as it's proved valuable multiple times for Cisco issues

[PhilipDAth]

No I don't, but it is pretty simple.

The first 24 ports are on ASIC1 and the next 24 are on ASIC2.

[cta102]

I assume anything below 24 ports is a single ASIC (or purely software based)

[PhilipDAth]

I don't know for the 8 port switches.  It wont be software based.  I would guess there would be a single ASIC.

[Kapil]

Hi Matthew,

Can you PM me the support case number? 

[Adoos]

Start at layer 1 and work your way up. 

9 times out of 10 it will be the patch lead, structured cabling or your wall outlet. I would suggest a decent fluke/netscout cable tester.

If your on site, connect direct into the switch port with a brand new patch lead. We have had bad batches of patch leads before also. 

[jbhehoman]

We had this same issue with a MS-120-48 port. There weren't any reported issues, but we swapped out a few cables and the errors were still showing up. Meraki ended up sending us a replacement and that device has been error free.