Need some advice (feedback?) re VLANs

[GavinMcMenemy]

HI All,

This might be a length post  - apols.

In a couple of weeks I plan on re-plumbing our network. This is something I've not had to do in a very long time and I've never done this using Meraki equipment.

Our corporate infrastructure consists of 2 sites that are geographically distant.
Our routers & firewalls are external and managed. The immediate routers are 2 pairs of Ciscos - I have no access to these or the core networking in this topology.
Both sites are completely Meraki.
Site 1:
A few Meraki APs (MR32 and a MR18)
6x MS 250-48lp - 4 are in a stack and service the user lan.
2x service our Servers and Hyper-V environment.
LAN is a single custom network. (10.120.x.x/20)

Site 2 (much simpler)
2x MR32 APs
2x MS350-48LPs
LAN is a single Class C network (192.168.1.x)

There are 2 24/7 SSIDs. We use Meraki to ensure SSID connectivity is the same in both locations. The sub office gains a SSID as part of their deployment.

For some of the year we have a sub-office on site who bring their own equipment and we provide infrastructure.

For this and other corporate security reasons I want to segment our networks into distinct LANs to serve various pieces of equipment and provide us with a method for providing this sub-office with infrastructure for part of the year.

What I am looking for is some advice: am I over-thinking this, are there better ways? What are the gotchas?

I'm not clear on how I route some traffic - read-on...

Note that, at this point, I cannot buy additional equipment.

Table notes:
I noticed that routing is stateless so to avoid confusion I've included that VLANs ID, I can't think of a reason why, in our environment packets, should not return to their own VLANs.

USERS + SERVERS, PRINTERS should be able to talk. Also needs access to the internet.
CS is the WIFI equiv of users so needs same access.
PRINTERS is the catch-all for MFDs which are shared. Our MFDs are managed by a secure print service in SERVERS. These do not need access to the Internet but should be accessible from both sites.
CS GUEST is a Guest network we use Meraki DHCP for that. It only needs access to the internet.
TEST is a proposed separate VLAN for testing purposes. Only needs access to the internet. No traffic on the rest of the LAN.
CS + USERS don't need to talk to each other but we're fairly liberal.

Our ISP is on board and will facilitate whatever infrastructure we need to build.

I've run up a diagram of the proposed network. Hope all is clear.

[Deviant]

Hi @GavinMcMenemy

This should not be to difficult to do, the biggest thing is the downtime you will need to convert users onto new subnets as well as servers and the effect it might have on some unknown variables.

The Internet VLAN seems confusing, I assume that that will be the VLAN between ISP and LAN. And when you refer to VLAN's allowed you mean to say VLAN X may Communicate with VLAN Y for example via the Routing Table of the switch.

All you would need is to have an IP in a VLAN on the Distribution MS250 side and also give the ISP an IP to use. Then they must route the relevant subnets to you're IP and you have a default route up to the ISP's router. The key off cause is to get rid of the /20 subnet mask on the end devices first or they wont know how to get to the other VLAN's. You would also need to ensure that you create the IP's for each VLAN to make them routable. Also ensure that you only tag VLAN's to Users and Server switches that are required on those switches. Choose a management VLAN for the switches and ensure on the trunk links that they are untagged.

Make sure you're switches receives DHCP in the management range VLAN and that it is permitted by the ISP's firewall.

As for controlling the flow of traffic, this would have to be carefully planned with the Switch-->ACL's. You would have to specify what subnet ranges can communicate to what subnets and apply that to the correct VLAN's

[GavinMcMenemy]

Hi Deviant,

I'm actually less worried about the users. We've picked the DHCP range to broadly present the DHCP range (with a different mask ...) that's present already. From reading the documentation we'd need to tag the ports in the USERS stack.

Servers though, you've hit one of my concerns. We've budgeted an entire weekend to work out the kinks. I've already placed Servers broadly into the range I expect them to be in so I hope there's not too much work to do there.

"The Internet VLAN seems confusing, I assume that that will be the VLAN between ISP and LAN. And when you refer to VLAN's allowed you mean to say VLAN X may Communicate with VLAN Y for example via the Routing Table of the switch."

Yes. Sorry I think I put that badly. the "internet" VLAN will contain a segment that we use to route traffic from the LAN to the external managed network. Traffic will flow site-to-site and to the Net.

"All you would need is to have an IP in a VLAN on the Distribution MS250 side and also give the ISP an IP to use. Then they must route the relevant subnets to you're IP and you have a default route up to the ISP's router. The key off cause is to get rid of the /20 subnet mask on the end devices first or they wont know how to get to the other VLAN's. You would also need to ensure that you create the IP's for each VLAN to make them routable. Also ensure that you only tag VLAN's to Users and Server switches that are required on those switches. Choose a management VLAN for the switches and ensure on the trunk links that they are untagged."

Great that's my thinking. We already have the IP for the router set up. One thing I am not clear on is the configuration of the route.

Example: If I am on VLAN 20 (users) and I want to route to the Internet my ISP will configure the necessary interface. But this raises a question for me. Do I make the default gateway for these devices the the one on VLAN 20 (e.g. 10.120.20.254) or set it as a the router address? I assume the former and if I am right about this I'd  just set the route to be:

VLAN 20 gw (10.120.20.254) -> VLAN 18 Internet gw (10.120.18.x)?

I find the Meraki route creation page a bit confusing.

"Also ensure that you only tag VLAN's to Users and Server switches that are required on those switches. Choose a management VLAN for the switches and ensure on the trunk links that they are untagged.

 Make sure you're switches receives DHCP in the management range VLAN and that it is permitted by the ISP's firewall."

I'm not sure what you mean there. I was just going to setup a helper address for Users DHCP and hand off to a microsoft server to facilitate the various services we've defined? 

And a final question re ACL creation. is there a best practise for when to do this? Right at the end or as we go along?

[Adam]

When you say both sites are completely Meraki do they have MX's as well or a different firewall?

[GavinMcMenemy]

Our

---

@Adam wrote:

When you say both sites are completely Meraki do they have MX's as well or a different firewall?

---

Maybe I wasn't clear in my long post.

Internally our LANs are all meraki. It's MS switches and MR access points. Our Routers are managed Cisco. I'm not sure what the firewall above is.

[Deviant]

Hi @GavinMcMenemy

As long as you can access all servers and change the subnet mask it should be fine since the actual IP wont change.

I guess there are two ways of doing it, the ISP creates all the VLAN's and tag them to you. In this case you dont care about any routing. But this would mean all traffic goes to ISP and back on 1 wire (Router on a Stick) which is not ideal.

The best way to do it is like you described for VLAN 20, when you create the vlan with 10.120.20.254. This becomes the default gateway for you're client in that VLAN. Then you have a static route to the ISP and the ISP has a static route to you in the ISP_Link VLAN for example. When a client wants to communicate to any VLAN's hosted on the Meraki Switch it would do a lookup on its routing table. When you host a VLAN it will be a directly connected subnet therefore it will know to send it to the next VLAN. When a packet is destined for Internet or other branch the switch will see the static route and know how to send it to the ISP due to the static route you configured. Don't be to worried as I use OSPF in this example, just look at the VLAN's and the default route at the bottom.

Once routing is in place the switch keeps a routing table which it refers to as per description above. A screenshot below to show the routing table on a switch, again ignore the OSPF for now. It will show interface when it is configured locally and static or default under route type. This will show you the next hop, notice there is no next hop for interface route types as they are local on the device.

Hope this helps.

Last thing I mean by receive DHCP for the switch, since the Meraki switch is cloud based it needs an IP from somewhere to get to the cloud. Unless of cause you do it statically in which case I guess that should be fine also, best bet might be to configure this in the so called internet vlan as you named it.

[PhilipDAth]

You really should choose a supernet for each site first, such as a /21.

For your main site, you could use 10.120.16.0/21.  This would give you 10.120.16.0/24 through to 10.120.23.0/24.

I would avoid using 192.168.0.0 for anything.  It is used to commonly, and if you ever need to build a VPN to anything you may run into overlapping IP issues.  Instead for your second site I would use the next /21.

For your second site, you could use 10.120.24.0/21.  This would give you 10.120.24.0/24 through to 10.120.31.0/24.

[GavinMcMenemy]

So.

We had this work scheduled at the weekend.

The routing was configured as described.

We decided to test this by switching over to the secondary router (it's a hot spare).

We allowed switchover to happen and I had a port ready to go to carry the vlans.

However no traffic would route to the internet. It took me and the support tech at our ISP a while to figure out the cause but in the Cisco logs we noticed a conflict.

The gateway address (10.120.18.7) was a duplicate. Looking at the MAC address the IP that was being advertised was the one for the MS250. We flipped back to the primary and everything reverted back to normal service.

This is how I created the first VLAN (Internet - 18).

Note that I binned the other VLANS for the purposes of this update. They were all created as outlined in my table above. The routing interface for each was the .1 for the subnet.

My ISP sent through a portion of the log which shows this:

This is what the CISCO (ISP) router saw.

Sep  7 2018 20:59:36.856 BST: %IP-4-DUPADDR: Duplicate address 10.120.18.7 on Vlan18, sourced by e055.3dfb.8d2e

I'm completely confused by this. Why would the Switch be advertising that it had the IP Address? Why would this go away when we flipped back to our normal routing arrangement?

And also yes, we lost our ability to manage the switch once we flipped over but I am not sure how to keep our internet connection alive during the change.

I have raised a ticket with support about this but other than an acknowledgement I've not heard back.

[Deviant]

Hi @GavinMcMenemy

I am sorry that you're change did not work out, but based on the configuration you supplied below. I agree with you it does not make any sense at all. I would wait for Support to respond with this.

As for the switch management being lost, that is the tricky one as this is a challenge. The switch can route via it's own routing table. Therefore you need another IP just for management. So you could give it a static IP also on VLAN 18.

If I could perhaps also ask, how did you interconnect to the ISP. Did you assign the switchport access vlan 18 or did you trunk some VLAN's and native vlan 18?

[GavinMcMenemy]

---

If I could perhaps also ask, how did you interconnect to the ISP. Did you assign the switchport access vlan 18 or did you trunk some VLAN's and native vlan 18?

---

The latter. The idea was to trunk all VLANs to the router.

This doesn't explain to me why the switch suddenly started advertising the gateway??

TBH: I'm having an issue with Meraki support. The reply I've had back is "You need to understand how the upstream router is going to work."

The thing is, I DO understand.

The router is going to listen on it's assigned IP address (10.120.18.7) and as I've outlined above it's expecting vlan tags to arrive via a trunk.

On the switch port connected to the router, I set the port to be a trunk that accepts all expected VLANs from this network. Or am I missing something?

Either way I've now exchanged 2 emails with support where I've sent them pictures of the setup and explained what I was expecting to do.

But they're not helping.

I don't know if anyone from Meraki support monitors this forum but I'm not getting the support I expect for this quite expensive kit. My colleague has separate issues with SM and essentially gave up trying to get support from Meraki. I hope I'm not getting more of the same treatment. I couldn't get hold of anyone on the phone yesterday.

/rant.

[Deviant]

Good Luck

Meraki Supports within a limited frame of work. It is very hard to explain use cases and customization, had the same issue with the management VLAN for Layer 3 Switches to the cloud with no resolve for months.

I was just curious about the VLAN tags, but you are right it would not be the reason. I agree to have it as an access port since this will be a routed link.

[GavinMcMenemy]

Hi Deviant I appreciated the advice and I'm quite happy to discuss the network we're proposing.

I don't know what Meraki was like before its Cisco acquisition (we only had APs and never needed assistance) but as we've now standardised our infrastructure around Meraki kit we've needed more assistance from Meraki. But every time we've talked to them we've had to abandon the ticket and muddle through because we get very little interactive pleasant customer service.