One Uplink, Two Firewalls. How??

[KirkinChicago]

This is something I've battled with for years, and I'm not sure the correct way to do this.  The internet connection that comes into my office is a single line.  But I have two Meraki MX85 firewalls, one is the failover, one is the primary.  If I want the secondary firewall to be able to seamlessly operate when the primary goes offline, how can I do this?  I've asked my ISP if they can provide me a second physical connection to the same network connection, and they said this would basically be a new circuit and would charge me double.  But if I plug in the connection to the primary firewall as I should, if that primary loses power, then there's no internet connection for my secondary.

I've called Meraki about this multiple times, created multiple tickets and tried many different things to fix this.  So far, the only solution anyone has given is for me to put a hub or some other switch in front of the firewall to essentially "split" the uplink to go to the two firewalls.  But if I do that, I'm adding more moving parts, and a single point of failire.

Okay, so I fudged the first paragraph to simplify my setup.  Here's more detail and my ACTUAL setup: I'm using two ISP uplinks, but both have this same issue where I need to "split" them.  I have two MX85 firewalls but one license, so they're primary/secondary and not HA.  I've tried even sending the uplinks to Meraki switch ports, assigning a random VLAN to three ports, then "splitting" the uplinks back to the firewalls that way.  But that gives me all sorts of STP errors, gives the switches incorrect public IPs, and I had all sorts of ARP flooding on some of the non-dedicated circuits.  It was just a nightmare and I had alerts coming dozens per day about the tunnels going down and coming back up.

So - without buying two standalone switches/hubs to split these uplinks, and without paying my internet provider thousands of dollars more a month to get a separate physical connection that I can feed to both firewalls to keep me operational during an outage... what is the correct way to make sure BOTH of my firewalls have BOTH internet connections without one firewall feeding off another?

Thanks - maybe there's a simple answer here, but it seems like the more than a dozen folks I've asked haven't been able to come up with anything.  And I say this almost every day to my team at work "there's no way we are the first people who had this issue, look it up or call support".  But I've tried so much, on something that seems like it would be an extremely widespread issue that almost everyone here would have had to deal with.  Why am I not able to find a simple solution for this?

Thanks, guys.

[KirkinChicago]

Oh, and I forgot to mention - we have been given a /29 network by our ISPs, so there are other public IPs I can use for the other firewall.  I just need to somehow split the incoming connections so that they feed both firewalls.  Not sure the best way to do this.  If there was a way to send ISP 1 to firewall 1, then send ISP 2 to firewall 2, then somehow bridge those connections to feed each other, that might work?  So both firewalls have at least one uplink they can rely on should the other firewall go down, but whatever firewall is primary can use both connections constantly.  If that makes sense.

Three offices, double MX85 pairs at each in primary/secondary role, Meraki VPN between sites, one Azure VPN and some VMs running in Azure, two ISPs at each office, each office has a switch stack of MS350's, the new office has C9300L switch stack.

Thanks again, guys

[alemabrahao]

The best, cost-effective solution for your situation would be to first have a link of at least a /29, then connect your link to a switch using a port configured in an access VLAN (e.g., VLAN 999) and a port for the WAN of each MX in that same VLAN.

But keep in mind that this isn't full redundancy, because if you experience a failure in this switch, the link to both MXs fails. Not to mention that you'd also need power redundancy for both firewalls, or at least a UPS.

https://youtu.be/UHfr90en9As?si=AZSiY9LfxummiXUY

[Mloraditch]

As stated, you need a switch, see @Ryan_Miles excellent presentation https://docs.google.com/presentation/d/1SBngZ5lBUa8fYSsIxhtxj2IYoKHwk4dZlNj7Wu3LcLU/edit?slide=id.p1... for setup options that provide varying levels of redundancy

[BlakeRichardson]

@Ryan_Miles that is a great document, I've never seen it before but it's very clear and easy to follow.

That is member of the month material right there!

[Shubh3738]

I have a quick question regarding the WAN breakout (single WAN, single switch, single MX) setup.

In this scenario, the ISP uplink is connected to Switch Port 1, and the MX is already connected to the switch through the SFP uplink port.

So, what’s the purpose of connecting Switch Port 2 directly to the MX again?

[RWelch]

Port 10 (switch) to Port 12 MX is in access mode VLAN 1000 (management).

Port 2 (switch) to MX WAN 1 is in access mode vlan 900 (ISP link up/down).

The management and ISP upstream/downstream are separated via access ports (is the difference).

[PhilipDAth]

I think the core issue is you want a particular type of service, but you don't want to pay for it.

You can plug ISP1 only into MX1, and ISP2 only into ISP2.  They'll be able to fail over.  This will work fine if you are not using a VIP, and you are only protecting outbound services.

It will also work fine if you are using AutoVPN, or a client VPN which uses the DDNS name.

There is also a dirty hack you can do where you can configure three of the LAN ports into their own dedicated VLAN.  Then plug ISP1 into one of those three ports.  Plug the second port into WAN1 on MX1 (it loops back), and the third port into WAN1 on MX2.

[KirkinChicago]

What folks are saying about taking the WAN links to the switch first, that's what I tried last time.  It was a nightmare.  My backup circuits at each site aren't dedicated CIDs, they're shared in the large buildings we're part of.  Sure, we get our own /29 IPs, but when I run these links through the switch, I get sooooo many ARPs that it does what I call "ARP flooding".  It causes my Meraki VPN tunnels to constantly bounce like 4 or 5 times a day randomly.

Also, I get a ton of RSTP errors and it keeps showing my switch's IPs as public IP, not the IP I've assigned it from the MX.  Basically, the switch starts using that uplink for itself instead of going through the MX.  

Overall, it was a very problematic setup.  And I don't want to have (or pay for) four total 1G WAN links.  I just need a way to "split" my two 1G WAN links to each firewall.  I would be fine feeding WAN1 to MX1 and WAN2 to MX2, but I don't see any way that my users and clients could use both uplinks simultaneously, so I'd basically be not using one WAN link constantly, which seems stupid.

Again, I would find it hard to believe no one else has had this same issue or hasn't come up with some solution.  There's no way I'm the only one who's been faced with this issue.

[alemabrahao]

To be honest, I've never had a problem with this topology; it's always worked very well.

In this case, I suggest involving your Meraki sales partner so they can better understand your scenario and assist you appropriately, along with one of their specialists.

[Mloraditch]

Same, I've also never experienced arp flooding or bounced vpns because of my sharing switch. I have seen where it will failover to using the providers dhcp for management if my lan management link is offline, but that I actually like as it provides me continued remote access in some outage scenarios.

This sounds more like your ISP has a strange setup and may need to tweak their handoff to you.

Sales may be able to help, but if you happen to be able to set it up even for testing with a spare switch a support case with the ISP and/or Meraki may be in order, to understand the problem.

[PhilipDAth]

This kind of topology has worked fine for me.