Creating HA vMX pair on GCP

[mraandrews]

Hey everyone, 

We have been trying to get a HA pair of vMXs running in GCP and to route traffic back to our internal network.  We have set up a Hub and Spoke in the Network Connectivity Center and created a cloud router and associated BGP sessions and configured this on the Meraki side too.

This all looks great until you try to make a connection to VM from the on-prem network.  What we see is that it seems to work and then not, almost like it is using the correct server (on its return path) to complete a connection in one request, but then the second time it is trying to complete the connection by routing back via the second vMX.

If we shut one of the vMXs down, everything works fine, and the routing table only contains one set of IP addresses (dynamically created), but as soon as the other vMX is up and running, it seems to create routes with the same prefixes, which would be fine, but the priority of all routes created by the BGP session is 0, according to GCP documentation, if this is the case it will use Equal Cost  Multi-Path  (ECMP) to determine the routes, which ultimately results in a load-balanced approach to routing.  So I guess my question is:

Why is the priority of these routes all 0, and why does there seem to be no way to change this in the BGP exchange from to Meraki to GCP (routing table), based on reading and online docs, I understand that the ANS Number can be used as AS-Path and can affect this, or is there another way to change this priority.

Any suggestions of help would be gratefully received.

Kind Regards 

[Inderdeep]

Did you follow this 

https://documentation.meraki.com/MX/MX_Installation_Guides/vMX_Setup_Guide_for_Google_Cloud_Platform...

[mraandrews]

Thanks @Inderdeep but I have not seen the article that you linked in your post, although setting up an individual appliance isn't a problem, it's making them a HA pair.

This is where I was given this link, to configure them using GCP NCC, but alas, this is where the issues of routing seemed to happen, and it used a round-robin approach that seemed prevent the ssh from our on-prem network getting to an VMs, or rather, getting back after the connection was made (as we could see the incoming connection on the VM, but it never returned to the computer that was initiating the connection).  

Anyway, turns out that the cloud router, using Hub/spoke and relying on BGP wassn't going to work for us, so I have now implemented this by manually specifying two routes that are the same, but the next hop is the each of the meraki appliances, and it the first one fails, the route becomes unavailable and the second route takes over, tiny amount of down time (around 1 minute), and Ideally I'd like these to work with no interruption.  SO any recommendations about that would be great.  I think the big issue was nto having any control over hte priority of the BGP priority when they were put into the routing table!  If you know a way to do that, then maybe I could try again!