vMX and Azure Firewall

[vMXNoob]

Hello All,

I am about to configure a vMX in azure - It will be in VPN concentrator mode - Are there any articles on integrating with an Azure Firewall so that we can secure the environment (The vMX will be effectively in front/inline with the firewall) as the vMX will not provide any firewall features in concentrator mode.

There is little to no documentation for this scenario - Also Im thinking that I am going to need a virtual network gateway in this scenario??

[Shubh3738]

Probably below link helps you:

vMX Setup Guide for Microsoft Azure - Cisco Meraki Documentation

[PhilipDAth]

If you want to use client VPN make sure you set the Availability Zone to none.  Even without client VPN, AutoVPN is more solid when not using an AZ.

[MartinLL]

I have done a few deployments that matches your scenario.

Here is the key takeaways.

• Before deploying the vMX, make sure that YOU create the network resources, not the vMX Wizard. When you do this you gain the ability to apply route tables and security groups to your vMX subnet. This is vital for your scenario to work.
• Place the vMX subnets in your hub VNET where your AZFW and/or Virtual network gateways are located.
• Create a route table that you will apply to the vMX subnet. This table should include all your VNET ranges. Keep in mind that you can not summarize, you must enter the VNET address ranges using the exact CIDR notation, then point those routes to your Azure firewall.
  Then create a new route for 0.0.0.0 with next hop internet. This guarantees that the vMX reaches the internet in the intended way, meaning through your instance bound public IP.
• Make sure that your workload VNETs (spokes) sends all traffic to the Azure firewall. In this scenario the AZFW will act as the Routing Hub for your setup. It is vital that the traffic flows through AZFW and vMX are symmetrical.

Now, there is two ways to solve routing from Azure to your Meraki Auto-VPN peers.

One is to use a routing table on the azure firewall. This is fine if you have a simple deployment with few subnets.

If you have a large deployment i would strongly recommend that you use Azure Route Server unless you are using Azure vWAN.

Place the Azure route server in your hub VNET and enable BGP in your Meraki Auto-VPN. Then add BGP peers to Meraki and ARS. Remember to add the azure route server subnet to Local Networks on your vMX in the meraki dashboard. If you do not do this BGP neighbors wont form.

Regarding your Virtual network gateway question. No you do not need it unless you want express routes. But i would still recommend it if you are planning on running 3. party VPN tunnels. VPN tunnels scales better on a Virtual network gateway rather then terminating them on your vMX.

Hope this helps!

Edit: When deploying the vMX i recommend that you select an Availability zone and not None. This changes your Public IP SKU from Basic to Standard. Basic PIP SKU is getting removed. This might save you some headache down the road 🙂

[vMXNoob]

This makes a lot of sense 🙂 - Clarification question - IF you don't mind please...

* When creating the VNet are you saying that the Subnets for both the vMX and AZFW should be in the same VNet?

[Shubh3738]

Why not using Auto VPN for the connectivity from your MX to vMX ?

[MartinLL]

Yes. This is due to azure routing logic with possible Vnet gateways and azure route server. If you designate a spoke as your vMX VNET and peer that to your hub VNET you will run into limitations with route learning. Best to pile network functions in the hub VNET.

[vMXNoob]

Hey Martin,

Question - I have had a look at the above and tested - I am not getting the BGP Peers to come up I have routed the vnets to the firewall and the default 0.0.0.0/0 to the internet - The routeserver subnet is added to the vMX so that they can form.

Any pointers - I can't find much reference documentation on these topics on MS..

[MartinLL]

Hey,

I see. First question. Did you add the route server subnet to the route table pointing to AZFW?

If yes, make sure to allow tcp 179 through the azure firewall.

If no, it should work by default unless you got security groups blocking.

Also, in the Meraki dash, make sure to sett EBGP Multihop to 5 or something. The route server is never 1 hop away in azure. If you route it through Azure FW you might need to increase the number. Increment by 1 until it works.

Check those first. If the issue persists check back in and we can look at it further 🙂

[Callum1]

Martin, i don’t suppose you’ve done a similar deployment where traffic is full tunnelled from the spoke site to the Vmx in azure, routed through the azure firewall and breaks out of one of the allocated PIPs on the azure firewall? 

[MartinLL]

No not in production. But in theory it should be possible. Route all traffic from the vMX VNET to the AZFW, but create a few entries for the Meraki stuff that exits locally. Like the Dashboard IP ranges and VPN registrar.

[vMXNoob]

Thanks for this - Is it worth as this is in lab to look at utilising vWAN instead get that set up and then add in a firewall and adjust the routing.

[MartinLL]

Depends on your architecture. But VWAN can get expensive fast. If you just plan on using meraki as a vpnc which acts as the only entrypoint to your environment i would say that VWAN is overkill.

Also, for existing azure environments the route server setup is easier to "shim" in without uprooting the entire architecture. Just my two cents 🙂

[vMXNoob]

Okay think that I am going to draw by hand (Quickly what I think the solution is) just to make sure that I am on the same page and you can understand my thinking. 

[vMXNoob]

Okay,

This is how I have things setup 

Azure

• vNET (Networking vNET)
  • Contains
    • AzureFW Subnet
    • Meraki SD Wan Subnet
    • Route Server Subnet
• Azure Firewall 
  
  • Rules for networking access to test device in server vNet
  • All vNets outside of networking VNet are peered to the networking vnet and also have route table pointing 0.0.0.0/0 to Azure Firewall

• Route Table
  • Associated to the vMX Meraki SD WAN Subnet
  • All Azure Vnets noted in correct CIDR format with next hop of Azure Firewall

• Route Server 
  • deployed with BGP Peers added
  • routes still not appearing in Meraki

Site and Meraki 

• vMX configured as Hub
• Site MX is step up as spoke vpn connecting to the vMX as the hub (default route not selected to go via hub so that we can do local breakout and only use vpn for wan destined traffic)
• vMX has BGP ARS subnet added as route

The only thing that I can ping currently from On site is the meraki vMX (thats coz i have a static route added for the vMX to check that the other side of the WAN was up)

I have attached badly drawn diagram - am I missing something??

[MartinLL]

Topology looks good.

Is the BGP peers up? You must add BGP peers to both route server IPs for BGP to come up.

How does your vnet peering for the workload vnet look? You need to tick a few boxes. The one where use peer gateway or route servers is the important one.

As for networks to add to your RT. Usually only workload networks peered to your hub. But you can point hub networks to the firewall as well if you want to firewall it. I usually choose to only add workload vnets.

[vMXNoob]

I am just in the process of scrubbing the environment again except the vMX - 

With regards to the workload vnet they are the only ones peer to the network hub. 

With regards to RT in the workload that is just routing to the firewall

I was more looking to confirm the RT in the networking (hub) vNet the RT that associates the vMX subnet what routes should be placed there pointing to the Firewall? 

[MartinLL]

I see that i formulated the answer badly. In your VMX RT you should only add your workload vnet prefixes and point them to azure firewall. I see that i mentioned all VNETs in my previous post... i meant workload vnets. Sorry about that.

[vMXNoob]

Let me test that will let you know the solution 🙂

[vMXNoob]

Hey - I have the BGP routes being advertised now - Whoop I can from the vMX subnet on a test machine ping devices on my lan side.

But cannot ping from my workload vm to lan side of things

Also cannot access the workload subnet from on premise even though I have the routes propegated 

Currently the firewall has an allow anything from my lan to drive out any FW issues. The workload vnet has a udr route to the firewall and is peered with the vnet containing the firewall

Any suggetions??

[MartinLL]

Great!

On a virtual machine in your workload VNET, can you select the network card and look at effective routes? Post the screenshot here and i will take a look.

This sounds like asymmetric routing through azure firewall.

[vMXNoob]

Here are the effective routes 

[MartinLL]

Is this the workload route table? Looks more like the VMX route table to me.

[vMXNoob]

That is the effective routes of a a server in the workload vnet

[MartinLL]

Ok, then you need to create a new RT, add 0.0.0.0/0 pointing to Virtual appliance and the AZFW IP.

Then set the option to propegate gateway routes to NO on the routing table.

Attach that to your workload subnets.

The reason (i think) as to why you can ping from a meraki spoke to workload, but no the other way is because from the spoke, traffic goes to the vMX, then to the AZFW. Traffic is then allowed and forwarded to the VNET and the VM. Return traffic does NOT go through the azure firewall and instead goes directly to the vMX.

When you initiate a ping from the workload VM this goes directly to the vMX and to the spoke. Return traffic is then send back to the vMX, but then is sendt to AZFW which does not have an active session for this traffic flow, which then results in traffic getting droped.

[vMXNoob]

Sorry I should clarify.

• Workload vNEt - No Ping to Meraki Spoke

• vMX - vnet - I have a test machines in a different subnet but in the same vnet (Effective route below) this can ping the Meraki Spoke

The Subnets in the vMX Vnet are 

• Meraki spoke (on prem) - can only ping the firewall and the vmx 

[MartinLL · Accepted solution]

Ok good. That means that Meraki and the Azure hub is OK.

You need to verify your vNET peering from HUB to Workload. Please attach screen shots of both ends of the peering.

Then you need to make sure that the workload VNET has the RT i described above.

[vMXNoob]

Workload Peering

vMX//Firewall Peering

If you want the full pictures happy to PM them to you