We currently have 1 single MX450 hub sitting inside our on-prem data center. A second and a third set of MX450 were acquired and after 2 months of design, tonight came down to bring them up and running. We've configured the LAN side to be in the same 10.28.0.0/29 subnet across the three sets of MX450, because it's easier to advertise via a single OSPF area 0 in the backend to Nexus 9k switches. Upon attempting to enable site-to-site VPN, on the second set of MX450, I received error message:
"There were errors in saving this configuration:
The local subnet 10.28.0.0/29 conflicts with a remote VPN subnet on the network IL - DC - CPC MX450-1 - appliance (10.28.0.0/29)."
Does this basically mean that all hub sites must sit inside diverse LAN subnets? And this translates to setting up multiple OSPF areas for route exchanges in the backend?
You basically answered my question. Layers 3 needs to be in different subnets, which means multiple OSPF areas.
We do not have a second data center, and Azure VMX cannot handle the traffic volume we have. And our existing MX450 hub cannot handle high volume of traffic when software patching is happening through 170 spokes, basically sheer volume of packets pegs the CPU of the MX450 to between 90 to 100%. I actually called Cisco out on this at their Chicago Cisco office. In any case, that's not my objective here.
Our company is expanding extremely fast and we need to scale out. Suggestions from multiple sources say to deploy multiple MX450 hubs. We are also looking at another SD-WAN solutions.
Well, we did consider using BGP, but it requires us to run MX450 in one-armed mode, which means we need a pair of super high-end high-powered routers in front of the MX450 to NAT with MX450 having just 1 single IP address. Since it needs to talk to the internet and internally at the same time. That was not an option for us either to scale out quickly, as it took some 6+ months for us to purchase a pair of Cisco ASR routers for a separate project due to supply chain issues. We just cannot wait that long.
There are things I wanted to do from design perspective but reality shows me otherwise.
So I'm left with this right now. Cisco responded last night and they can disable the ability for hubs to talk to each other in the backend, it's an hidden option. I'll be calling Cisco support this morning to get this going.