Multiple MX450 hubs - LAN side must be different?

[ronnieshih75]

We currently have 1 single MX450 hub sitting inside our on-prem data center.  A second and a third set of MX450 were acquired and after 2 months of design, tonight came down to bring them up and running.  We've configured the LAN side to be in the same 10.28.0.0/29 subnet across the three sets of MX450, because it's easier to advertise via a single OSPF area 0 in the backend to Nexus 9k switches.  Upon attempting to enable site-to-site VPN, on the second set of MX450, I received error message:

"There were errors in saving this configuration:

• The local subnet 10.28.0.0/29 conflicts with a remote VPN subnet on the network IL - DC - CPC MX450-1 - appliance (10.28.0.0/29)."

Does this basically mean that all hub sites must sit inside diverse LAN subnets?  And this translates to setting up multiple OSPF areas for route exchanges in the backend?

thank you.

[Brash]

Yes (to some extent). You can't have all 3 hubs advertising or receiving advertisements for conflicting subnets.

They have to be in separate L3 domains.

What was the design decision around purchasing 3 MX450's for the same datacenter location?

Are you planning on using different MX's to advertise different subnets?

Or are you using it for failover purposes? - In which case you might want to look into setting up a warm spare.

[ronnieshih75]

You basically answered my question.  Layers 3 needs to be in different subnets, which means multiple OSPF areas.

We do not have a second data center, and Azure VMX cannot handle the traffic volume we have.  And our existing MX450 hub cannot handle high volume of traffic when software patching is happening through 170 spokes, basically sheer volume of packets pegs the CPU of the MX450 to between 90 to 100%.  I actually called Cisco out on this at their Chicago Cisco office.  In any case, that's not my objective here.

Our company is expanding extremely fast and we need to scale out.  Suggestions from multiple sources say to deploy multiple MX450 hubs.  We are also looking at another SD-WAN solutions.

[ww]

I like this design more

https://documentation.meraki.com/MX/Deployment_Guides/VPN_Concentrator_Deployment_Guide#Deploying_a_...

combined with bgp to a layer3 core, you can use multiple concentrators and from the spokes  you select which concentrator to use.

[GreenMan]

Yes - the limitation on advertising exactly matching subnets applies to Hubs in routed mode;   https://documentation.meraki.com/MX/Networks_and_Routing/MX_Routing_Behavior#Routed_Mode_and_AutoVPN  VPNC mode is more flexible in that way.

In passing;   in VPNC mode you can also run eBGP as an alternative to OSPF - it's a lot more powerful:  https://documentation.meraki.com/MX/Networks_and_Routing/BGP

[ronnieshih75]

Well, we did consider using BGP, but it requires us to run MX450 in one-armed mode, which means we need a pair of super high-end high-powered routers in front of the MX450 to NAT with MX450 having just 1 single IP address.  Since it needs to talk to the internet and internally at the same time.  That was not an option for us either to scale out quickly, as it took some 6+ months for us to purchase a pair of Cisco ASR routers for a separate project due to supply chain issues. We just cannot wait that long. 

There are things I wanted to do from design perspective but reality shows me otherwise.

So I'm left with this right now.  Cisco responded last night and they can disable the ability for hubs to talk to each other in the backend, it's an hidden option.  I'll be calling Cisco support this morning to get this going.