Weird connectivity issue /w switch & SFP+ from other vendors

[jsathaseevan]

I have an MS225-48 and an MS225-48FP stacked. They are working great.

They have replaced 3 x Netgear S3300-28X switches that we used in our main closet. These three switches have now been moved to 3 new buildings. Everything works great over copper and 1GB.

We had a vendor come in and run us some fiber so we could make use of the 10GB connections on both sets of switches. 

For the building I'm occupying, I've used 2 x Tripp-Lite N286-10G-SR-S 10GB SFP+ on both my end, and the Meraki end in the main building. Works GREAT. Meraki dashboard tells me everything is A-OK and connected at 10GB Full duplex. Netgear switch tells me the same.

The next building, same configuration, will not give me a connection (except over copper). Same part numbers. I've had the vendor come back and test the lines (I can see light as well), and I've even removed the very SFP+ modules connecting my building to try and connect the new building. Nothing. I've switched out the patch cables, also using the ones from the working configuration. 

I'm at a loss for words. Meraki support has informed me that as the SFP+ modules aren't their nor are the switches on the other end, they can troubleshoot no further.

Any ideas?

[ww]

Swap 1 switch from stack to other building 

[PhilipDAth]

Are you getting no link light?

Number #1 issue - you need to swap the send/receive fibres over.

[jsathaseevan]

Thank you for your reply.

No link light.

Indeed I switched the send/receive fibers around. I ensured there was light passing through the entire drop and patch cables...and that the lights did not collide.

[jsathaseevan]

Thanks for that. Simplest thing I haven't tried yet. I swapped the switch...the working switch from my building went into the new building.

STILL NO JOY! I'm thinking there is simply an issue with the cabling and I will have to find another vendor to come out and check the issue

[Bruce]

What length is the fibre run? What fibre is it, single-mode (OS1, OS2), or multi-mode (OM3, OM4)?

[jsathaseevan]

Thank you for your reply.

The fiber run is OM3, multimode, <100m. I have a copper run that is currently allowing the building to have Internet access. 

[PhilipDAth]

Another thing you could try is to move the switch to right next to the core (temporarily) and connect them using a fibre patch lead.  If that works, you know it is not the kit or SFP's.

[jsathaseevan]

So I did this as well. I actually didn't use the down switch, but instead used a Meraki MS-120 to test and a 12ft fiber patch. It wouldn't light up either, but I was told that that was due to the fact that the MS120's don't support SFP+. I'm going to be forced order two Meraki SFP modules to do any more troubleshooting it seems.

[Bruce]

Did the installer provide you a test report showing the loss across each of the fibre cores from end to end? It sounds like a fibre problem, but as Philip said move the switch and test with a patch lead, that will pretty much confirm if it’s the fibre run or the kit.

[jsathaseevan]

Hey Bruce, They did not provide a report. But they did charge me $400 to come back in and test the line!

I will request it...though I don't know how to read it....maybe the community can help me make sense of it.

Thanks

[RobinGanderton]

Curious if you found out what exactly the issue was.

I'm experiencing the same thing.  I have 2 of 3 10 Gb backbones in place.  One of the 10 Gb Fiber backbone will NOT connect.  SFP's have confirmed working (Moved both, over to the working backbone), fired right up.  Took the working backbone SFP's to needed link, of which will not start.

The dashboard sees the SFP's.  I've even confirmed the fiber using some old 100 Mb d-link media converters.

I had read sometimes you must Force those ports to 10 Gb, but that is not even an option.  I can do 1 Gb auto or 1 Gb Forced.

Anybody else having said similar issues ?

[Bruce]

@RobinGanderton if the modules worked in other devices and another link, and are being correctly reported as you state then I’d tend to go with it being a fibre problem. The characteristics (e.g. dispersion, scattering, attenuation, and ultimately loss) required of a fibre link to support a 100Mbps link are very different to those required to support a 10Gbps link, for example a 100Base-FX module can drive 2km on a good OM3 cable, but a 10GBase-SR will only get about 300m on the same cable. You should probably find someone with optical test gear to run a test on the link to see if it is able to support the 10Gbps link.

[RobinGanderton]

Thanks for the Fibre idea.

The 10 Gb Fibre had lit up the week before.   I confirmed the Fibre backbone about a week prior.  I wasn't ready to use it yet, and the switches were still configured OK, so I simply shut one of the ports off (Dis-abled)

Week later, confirmed the fibre again with the media converter, put all cables where I wanted them, and hooked up.  No link.

I did read swap fibres, which I haven't done.  I was going to go in the management port to take a close look at the port but could not get logged in the management side.

The run(s) themselves are < 100 meters.  Nice clean Corning Fiber shelves (6 pair a piece;  ALL tested fine) OM3;  Brand new patch SC-SC; Tested. etc.

Just odd, that it fired up fine, well a couple weeks ago, and now not.

[jsathaseevan]

Hey Robin,

I have not resolved this.

My scenario is similar to yours. I have one main building and 3 outer buildings. 

Fiber was run from main building to:

- IT trailer

- a mechanics shop

- a purchasing trailer

I now have 2/3 buildings running on fiber, with the last having to use copper @ 100Mbps because we can't get the fiber running. 

I've moved the SFP+ modules around after hours on working links and confirmed that the modules are fine. 

I've called the fiber install to come in and they've stated that the fiber is fine.

I can only assume I have a busted switch. I've gone into the switch's control panel and confirmed that the fiber ports are enabled. 

My dashboard however does NOT see any of my SFP+ modules. I believe that I will need Meraki modules instead of the Startech ones I'm using to see the modules. 

I might just have to do that in order to continue troubleshooting.

I get NO link at all. I do not get reduced functionality

[RobinGanderton]

Morning All!

K, here was my issue (Fibre backwards).  I put another 1 Gb SFP on different switches in the same area.  (Another switch in each stack)

I know the 1 Gb work, because I was using them for imaging.

I finally tore a SC connector apart (Very easy) and swapped fibres.  Came right up.  At least at  1 Gb

I must have had only one crossover, and must have used it on the 10Gb trunks that work.

If you wish to duplicate, (Move fibre) do so gently.  Watch your optics.

I hope that helps others.

[jsathaseevan]

HI Robin, I've attempted this without any luck.

I have SC connectors between for the fiber patch panels of the buildings.

I use an SC-LC jumper to connect from patch panel to switch. I can easily switch the polarity without taking apart my cables. I ensured that I can see light coming from one side and made sure that the light going back is on the "dark" side.

switching it back and forth has no effect for me.

One thing I failed to mention:

The fiber going between each building happens to be a 3 pair fiber cable. I have tried all 3 pairs to send my signal across. Indeed I see light on all three pairs. I'm beginning to thing there's something wrong with the switch itself. It has two fiber ports. Neither lit up my connection, though BOTH sent light down the fiber jumper cable.

I'm going to have to bite the bullet and replace that switch.

[RobinGanderton]

Darn, not sure what to suggest.  The switches are not ? recognizing the SFP s ?

If not, First thing I'd do is try different ones.  (If you got)

I was a little lucky.  Had some old dlink media converters to test with.    That confirmed fiber.  (Mind you, techs with good eyes might even see the light - grin)

The only other thing I could suggest, is try the SFP's in other switches.

[jsathaseevan]

So I have Startech and Tripplite SFP+ modules. They both worked (mixed) in my 1st link from server room in main building to my IT office. 10GB no problem.

The configuration is as such:

Meraki MS220 switch -> StarTech SFP+ module -->--> Tripplite SFP+ module -> Netgear ProSafe Switch

When the two new buildings went up and fiber was run to them, I bought 2 each of the same StarTech and Tripplite modules. Those did work to get another building up. They did not work to get the 3rd building connected. 3rd building is currently using copper until I can figure it out.

My inventorying is bad. But I think I've found a spare switch to try. I"ll be able to try this on Thursday when back

[Eddy_X]

I had a similar issue before which would not show connection even the fiber has laser.

In my case, the fiber between switches is 62.5/128, the orange ones. 

I was using MA-SFP-10GB-SR and the weird thing is that there was only one connection that worked, and not the others.

I end up using an old SFP Cisco DS-SFP-FC8G-SW on both ends and I notice the link is 1G rather than 10G or 8.5 G.

Maybe you want to try force 1G with full duplex on the uplinks. 

[RobinGanderton]

Just a thought, back to the begininng of this thread.

Have you spoken to Cisco direct ?

[jsathaseevan]

No, Meraki only.