Adding switch to external building via fiber link

[agionet]

Seeking some advice on how best to expand our network into an external building on a university campus.

Currently our topology is like this:

Essentially just a MX84 firewall connected to an AARNET Network Termination Unit, a couple of L2 switches, mostly going to desks, and some Wireless AP's throughout the building.

The university campus staff advise that they can "patch us into" the other building via fiber optic, which is daisy chained through a couple buildings, and that we should just be able to put in a switch at the other building with a trunk for all traffic.

There should only be about 8 x staff at this other building at any given time, and there are patch panels in the existing rack to connect our switch to the desks.

I've never done this type of design or worked with a fiber connection before, so seeking any and all advise on how to set up and configure this and get it working.

[alemabrahao]

First I think you have to check the type of fiber, if it is monomode, multimode, etc.

If you have doubts, it is best to hire a specialized workforce.

[agionet]

Hey, thanks for the reply.

The fiber link between buildings is 12 core SMF.

[amabt]

as alemabrahao has mentioned. They key here is to know what type of fibre and get the apporirate module for the switches on both end.

[agionet]

0s2 12 Core SMF with LC-LC connectors, from my understanding we should use something like this:
Cisco Meraki MA-SFP-10GB-LR 10GBASE-LR SFP+ Transceiver

[Brash]

Correct, a 10GB LR transceiver will match the fiber you've described.

You can also use any of the other SMF options listed in the following link (assuming your switch supported it).

SFP and Stacking Accessories - Cisco Meraki

It sounds like a fairly simple topology - just make sure your VLANs are trunked in all the right places.

If I'm understanding you correctly in that your network is logically separate from the University campus network but is transiting across it, you might want to consider placing an L3 switch in the new building. You can then place the clients there on a separate L3 network and use a transit VLAN to connect between the new building and your existing network (through the university switches).

If that's not correct and you are a part of the university campus network, it's still a good practice to split the L3, but you could also just trunk your L2 VLAN's to the new building switch.

[agionet]

Thanks for the detailed response.

I realize I'm probably making a fool of myself for asking such simple questions, but I'm still relatively new to networking and have never used Meraki gear or fiber connections prior to inheriting this network with zero documentation about 3 weeks ago.

You are correct, we are separate and have our own ISP link at our building but are piggybacking off the campus fiber links between buildings to expand our LAN into another building.

I'm planning to use a MS250-24P L3 switch with the above linked transceivers.

Currently we only have one primary VLAN (All ports on our current primary switch read "Trunk: Native VLAN1) for the internal network, there are others in use but won't be required for the staff in this secondary building. 

Can I clarify what you mean by trunking in all the right places?

My plan is basically this:

[Brash]

Not at all, that's what the community is here for.
We've all taken over undocumented networks in our time.

I think I now understand - The university is providing a 'direct' patched fiber connection to the new building. It doesn't run through any of their switches.

In that case, it's easy. If you're only using VLAN 1, you just make sure the port connecting to the new switch is configured as a trunk allowing VLAN 1 and plug everything in.

As a side note, using VLAN 1 is typically seen as a poor design. In future you may want to look at creating other VLAN's and segmenting traffic.

[agionet]

Late reply as the switch only just arrived - so we should configure the SFP port on our primary switch as a trunk port for VLAN 1, then hook up the fiber links. Do we need to then configure the SFP port on the new Layer 3 switch as a trunk port, then configure all other ports as access ports for VLAN 1?

[amabt]

You need to make sure the VLAN allow list matches on both side. Both side needs to be trunk port.

[agionet]

Hey guys, I'm having an issue linking the two switches via fiber. 

I've installed the SFP modules and they show up on in each respective port in the meraki dashboard, but there's no link being established.

I tried inverting the polarity of the fiber where it comes out of the patch panel in the second building and it linked for a few seconds, was able to see the switch in Meraki dashboard but then it suddenly dropped off. Have tried other ports and swapping it around again to no avail, unable to get it to link up again. 

We've run a cable test on the fiber link and it's OK but no matter what SFP port or transceiver we use it won't link. I have tested the same SFP ports using RJ45 transceivers and they work fine. 

Any ideas? I did notice some RSTP events in the event log in between the time the link first came up and before it dropped off. The new switch shows as it's own root for RSTP but all of the other switches in the main building have the primary switch (the same one that the fiber link is going to for the switch in the second building) as their root. Is there some configuration I need to make to get this to work? From my understanding I should at least be getting a link light coming on so I'm not sure what would have caused it to work briefly then suddenly drop off and not work again.

[JacekJ]

How does the port config look like on both sides?

I understand you don't get an uplink LED on these ports?

What is the distance, what are the SFP models?

[agionet]

Both ports are set to trunk with all vlans allowed, native vlan is the same on both.

The link LED came on after inverting the polarity of the LC fiber in the patch panel of the second building. At that stage I could see it connected in meraki dashboard and locally via the management port of the new switch. This dropped after a few minutes and I've been unable to get it back after trying different SFP ports and rebooting the switch several times.

Distance is maybe 150 metres max.

The SFP transceiver modules are MA-SFP-10Gb-LR

[agionet]

As another update to this, I managed to get the link working by forcing both ports to 1Gbps. 

From my understanding we should be able to use 10Gbps as both switches have SFP+ ports, and we're using SFP+ transceiver modules. 

Can anyone shed some light on why this only works when it's forced to 1Gbps fdx?

[JacekJ]

You need to find out what type of fiber is used and if its OK, the distance you said max 150m.

If its an old infrastructure then this might be explaining the issues (like OM2).
https://documentation.meraki.com/General_Administration/Cross-Platform_Content/SFP_and_Stacking_Acce...

[agionet]

The fiber is OS2 not OM2

[JacekJ]

Sorry, I somehow missed the part where you wrote about the fiber type above.

Then I suppose there is something in the SFPs or the fiber, that it can't negotiate higher speeds.

Without proper tests it will be hard to tell.

You could switch around patch cables, SFP ports and modules if you have an option to check different paths.

You have any option to test the SFP on a smaller distance in the cabinet? I sometimes even create loops from one SFP port to the other with a short patch cable, for a second, just to see if things work, if they negotiate proper speeds, so I know that both SFPs and patch cables have no obvious problems...

[Brash]

What model are the switches?

[agionet]

They're both ms250-48lp

[Brash]

Ok, they're both capable of 10Gbps.

Sounds like it could be an issue with the fiber then. I would do a loop back test on both sides to rule out the switches and SFP's 

[PhilipDAth]

Check the SFP compatibility matrix to see what is compatible with the switches you are using and the type of fibre.
https://documentation.meraki.com/General_Administration/Cross-Platform_Content/SFP_and_Stacking_Acce... 

For example, you might be able to use much cheaper 10GBASE-LRM (as opposed to 10GBASE-LR) if you are going less than 300m.  Also, do you need 10Gbe for 8 users?  1Gbe is much cheaper.  You could check out 1000BASE-LX10.