MX100 HA issue

SOLVED
Narnia534
Here to help

MX100 HA issue

We have:

2 MX100's setup in HA mode

2 ISP's

2 MS425 Switches stacked

1 MS120 switch between the ISPs and the MX100s

 

The MS120 gives us signal from our ISPs to both MX100 (different IP addresses of course)

As far as I can tell, the MX100 are working in HA mode. And both ISPs are working.

 

Narnia534_0-1591268259913.png

Narnia534_1-1591269017178.png

 

On MX1

Ports 1 & 2 are connected to ISP1 and ISP2.

Port 9 is connected to an onsite web server (configured as a DMZ)

Ports 8 and 11 are each connected to a different one of the stacked switches.

So far so good

 

On MX2

Ports 1, 2, & 9 are the same as above.

However, as soon as I try to connect port 11 to one of the stacked switches, our Internet goes down instantly.

Since there is no connection at that point to the cloud, I can't capture packets from that port, can't see what is happening.  As soon as I disconnect it from the stacked switch, Internet is back up.

 

Can anyone give me a steer as to what to look for, or what I'm doing wrong?

Many thanks

 

1 ACCEPTED SOLUTION
ww
Kind of a big deal
Kind of a big deal

The second  solution i gave you should  also work

View solution in original post

22 REPLIES 22
ww
Kind of a big deal
Kind of a big deal

At the switch currently  1 Port is stp blocking?

 

How are the ports that connect mx and ms425  configured ?

 

Narnia534_0-1591270431602.png

 

Nick
Head in the Cloud

You are not tripping over the anti spoofing on the MX?

 

Static routes etc all in place?

 

https://documentation.meraki.com/MX/Firewall_and_Traffic_Shaping/IP_Source_Address_Spoofing_Protecti...

rhbirkelund
Kind of a big deal

Is it correct to assume, that the MS120 acting as a "wan switch", is connected back to the LAN side, to allow management?

Could you perhaps try to draw up the topology?

LinkedIn ::: https://blog.rhbirkelund.dk/

Like what you see? - Give a Kudo ## Did it answer your question? - Mark it as a Solution 🙂

All code examples are provided as is. Responsibility for Code execution lies solely your own.

The MS120 is not connected back to the LAN. It is connecting to the cloud via the ISPs, and I can manage it from the Meraki Dashboard

 

I'm not very good at topology, so here is an attempt:

The MS120 has 3 VLANs connected to it: 110, 120, and 95

The Web server has 2 NICS in it, one for VLAN95 and one for VLAN1. This VLAN1 is connected to another MS125 switch which is plugged into both core switches

Each of the desktops is on VLAN1

 

As you can see, MX2 is not plugged into the LAN at all, because when I do, it stops our Internet. That's what I'm trying to correct.  MX1 is plugged into both core switches, with STP blocking one.

 

 

Narnia534_1-1591280427721.png

 

 

ww
Kind of a big deal
Kind of a big deal

Can you disconnect  Port 9 of mx2. 

 

And then try  connect mx2 to your stack again

 

 

Or

 

Connect the server at vlan 95 to the stack  and add vlan 95 to the trunk to the mx. (And Disconnect both 95 cables of the ms120)

I can try that tonight when most people have gone home.

 

I'm not sure I understand your 2nd option.  Vlan 95 is supposed to be a DMZ.  If it is going through the stack, wouldn't that eliminate the safety of the DMZ?

ww
Kind of a big deal
Kind of a big deal

If you connect it using layer2  vlans to the mx its no different then now.  The  layer3 interfaces  stay en on the mx.  Vlan 1 and 95 can only  communicate  when the traffic passes the mx (or not with firewall rules  in place).

 

Proposed  scenario 1 is the quickest  to test. If that works, its probably  best to config scenario  2 as solution

cmr
Kind of a big deal
Kind of a big deal

Personally I would ditch the MS120 on the ISP side and get a pair of unmanaged L2 switches, Cisco do some small ones that have been reliable for us. As for the DMZ, I'd use another separate switch for that, or if only the one host, directly cable both port 9s to the webserver.  You could use the MS120 for that role

 

Sorry, forgot to explain my reasoning, which is that is the MS120 goes down you lose everything.  The purpose of two ISPs is normally to increase availability and by putting them both in one switch you have largely negated that benefit.  Secondly having outside and DMZ on the same switch is not 'clean' and could easily lead to the wrong port being used for the wrong connection.

Nick
Head in the Cloud

Not to go off topic - but how would you propose those layer2 ISP switches are setup @cmr 

cmr
Kind of a big deal
Kind of a big deal

@Nick no config, use unmanaged switches, we use 5 port models with MX WAN ports occupying two and ISP drop cable in the third.  Two spare ports to use for diagnostics. 

 

As for the MS-120 I'd have it all on one VLAN and it can reach the Meraki cloud by going though port 9 on the MXs and then out to the internet.

 

When dealing with outside/DMZ I always like to keep it simple as when it matters, it won't be you there, it will be someone like a security guard on the end of a phone who may not understand the complexities of the setup. 

Nick
Head in the Cloud

@cmr 

 

Ah I see what you mean now - I wasn't reading the previous properly

 

I concur with that, keep the upstream nice and simple in you can as CMR says its easy to figure out what is what and you don't benefit much from have an MS in play there

Good call, @ww , when I disconnected Vlan 95 from MX2, then I could successfully plug in to VLAN1 on MX2.

 

So why is that an issue? Shouldn't STP on the MS120 have stopped it?

Or is it possible because the WebServer only had 1 IP address?

This is still not solved.  Been working with Meraki Tech Support, but so far nothing

 

What works:

  1. MX Warm Spare - High Availability Pair
  2. Two WAN uplinks

What doesn't work:

  1. DMZ (VLAN95) connected to both MX devices

 

The problem, as I understand it from Meraki, is that the MX Firewalls do nothing with STP.  And the MS Switches use RSTP, but do not deal with STP on a per VLAN basis.  As @ww saw, as soon as I unplug the DMZ VLAN from the secondary MX, everything else works.  Apparently VLAN95 connected to both MX devices completes a loop and STP shuts down VLAN1 from the Primary MX.

 

One solution, they tell me, is to replace the WAN Breakout MS switch with a Cisco Catalyst switch which has PVST or Per VLAN Spanning Tree.  This would isolate VLAN1 from VLAN95 STP wise

 

Can anyone think of anything solutions?

Thanks

ww
Kind of a big deal
Kind of a big deal

The second  solution i gave you should  also work

Thanks, I am definitely looking into this further.

Thanks @ww . That did take care of it.  Meraki support kept trying to find a way to make my solution work, but in the end he agreed with you.

Thank you very much

Nick
Head in the Cloud

So your revised setup looks the same - with the VLAN 95 connected into the MS Core instead of the MX units themselves?

 

Twin connections from each MX into the MS Cores as well?

Below is what it looks like now (hopefully a better drawing):

  • That port on the core is now configured as Access, Vlan 95. 
  • Nothing else on the Core is configured for Vlan 95 (no routes, no ACLs. It just goes to the MX),
  • But the MX is configured for V95. 
  • STP disables the redundant ports on the Cores

 

Narnia534_0-1594301682397.png

 

PhilipDAth
Kind of a big deal
Kind of a big deal

>Ports 8 and 11 are each connected to a different one of the stacked switches.

So far so good

 

Connect each MX with a single connection to the switch stack.  Then you remove the L2 loop.  Then it will work and be reliable.

Unfortunately that was not the problem.  I started out that way with just one connection to the stack from each MX.

I still had to disconnect MX2 to have Internet.

rhbirkelund
Kind of a big deal

I have previously done something like this. Some of the elements could match yours. It might not be the prettiest, but under the circumstances that were given to me, it works. Been running for the last two years, and so far haven't heard anything from it.

Ignore the blue line. It was for a telephone service provided directly to the LAN.

 

In your case HX1-SW1 would be your MS425 stack.

 

Skærmbillede 2020-06-05 kl. 09.38.13.png

LinkedIn ::: https://blog.rhbirkelund.dk/

Like what you see? - Give a Kudo ## Did it answer your question? - Mark it as a Solution 🙂

All code examples are provided as is. Responsibility for Code execution lies solely your own.
Get notified when there are additional replies to this discussion.
Welcome to the Meraki Community!
To start contributing, simply sign in with your Cisco account. If you don't yet have a Cisco account, you can sign up.
Labels