Meraki vMX & AWS security group

[Stallone]

Hello 

Can someone help me resolve my issue with the AWS security group on the vMX instance?

I have a connection from my on-premise network to the AWS VPC via Meraki vMX, and everything works fine.

However, I would like to apply some firewall restrictions to the traffic coming from on-premises to the AWS VPC.

I tried to apply a security group that allows only SSH inbound traffic and all outbound traffic to the EC2 vMX instance. For some reason, it didn't work for me. I am still able to ping my resources in the AWS VPC from my on-premises network.

Also, I would be grateful if someone could explain how the Meraki vMX connection with the EC2 instance works under the hood. At this moment, I thought that all traffic coming from the on-premises resources to the AWS VPC first faces the EC2 ENI interface. Why didn't my security group block ICMP packets?

[alemabrahao]

Maybe it can help you.

Work with security groups - Amazon Virtual Private Cloud

[Stallone]

Unfortunately No

[PhilipDAth]

The EC2 security group (which you could apply to the VMX) has both inbound and outbound rules.  It absolutely controls what traffic is allowed in and out of an EC2 instance.

You could also consider using  Meraki AutoVPN firewall rules - which is likely to be simpler.

https://documentation.meraki.com/MX/Site-to-site_VPN/Site-to-site_VPN_Firewall_Rule_Behavior 

[Stallone]

Yes, I had the same thought, but unfortunately, inbound traffic is being ignored for some reason.

Below is a screenshot of my Security Group's inbound rules; as you can see, it's empty.

However, for outbound traffic, I have a rule that allows all traffic.

For example, if I try to execute a ping from my 10.56.33.0/24 subnet to an AWS resource located in the same subnet as the vMX instance, I can reach it, and it responds.

However, the responses can only reach my host if the outbound rule from the screenshot is applied.

[MarcoS]

Hi @Stallone,

Are you pinging the public IP or the private IP of the vMX? Are you pinging from a host across the VPN tunnel?

If you are pinging the private IP of the vMX through the VPN tunnel, then look into the VPN firewall rules as shared by Philip in one of the replies, you can block ICMP there. After you add the rule in the dashboard, give the MX a few moments to ensure the config is updated ('Up to date' status) on the device.

"I am still able to ping my resources in the AWS VPC from my on-premises network."

My understanding is that this is expected as you are tunneling traffic into the EC2 (effectively a software VPN termination inside the EC2 instance).

Try this:

- ping the private IP of the vMX from another EC2 instance in the VPC (i.e. not a host not across the VPN tunnel), your Security Group should work. 

- ping the public IP of the vMX from your on-prem, your Security Group should work.

Remember that if you are testing with a running ping (i.e. the ping is already running and only after that you block traffic, stop and restart the ping to validate traffic is blocked, else the ping will continue because of the active flow).

"if I try to execute a ping from my 10.56.33.0/24 subnet to an AWS resource located in the same subnet as the vMX instance, I can reach it, and it responds." "However, the responses can only reach my host if the outbound rule from the screenshot is applied."

My understanding is that this is also expected as if you block the outbound traffic, the EC2 instance hosting the vMX  cannot send those packets out. You mention "it can reach it, and it responds" but have you checked that the ICMP request actually reaches such resource? For example, if that other resource is an EC2 instance, run a tcpdump on it, you should not see the ICMP Requests at all (because they would be blocked outbound from the vMX EC2 - if you remove the outbound rules).

Let me know how it goes.

[Stallone]

Hello @MarcoS 

Thank You for the support

I tried to do what you said, and yes traffic from the another EC2 instance was blocked

Could you please explain why traffic that goes through the VPN tunnel from on-premises resources is not blocked by the vMX instance's security group, while traffic going from AWS to the on-premises resources is blocked?

[MarcoS]

Hi @Stallone,

I will share my understanding:

"Could you please explain why traffic that goes through the VPN tunnel from on-premises resources is not blocked by the vMX instance's security group"

It is not blocked by the Security Group because of the very nature of VPN. The vMX is a software that acts as the encryption termination point for your VPN tunnel, inside the EC2 instance; this means the ENI of the EC2 instance only sees encrypted traffic (in this case, Auto VPN traffic, which uses UDP encapsulation, choosing UDP ports in high ranges, 32768-61000 last I checked).

Try this:

- Setup VPC Flow logs for the vMX ENI. Pick the destination you prefer (CloudWatch Logs or an S3 bucket, depending what the type of permissions you have in AWS). Now ping through the tunnel (for example, from on-prem to an EC2 instance behind the vMX inside the VPC - or anything through the tunnel, in whichever direction). Check your VPC Flog logs, you won't see any of those ICMP packets (except the vMX itself pinging 8.8.8.8 for connectivity checks, which happens outside the tunnel, so you will see those), you will only see the outer encapsulation, meaning UDP packets on high port ranges which are for the tunnel between the vMX and the on-prem MX, as well as packets towards the Meraki VPN Registry on UDP 9350-9381.

As an analogy, it's a bit like you establishing a VPN from your laptop (representing the vMX) towards somewhere on the internet (representing your on-prem MX), your network firewall (representing the EC2 ENI) won't see what you do inside the tunnel, will only see the outer encapsulation.

"while traffic going from AWS to the on-premises resources is blocked?"

Can you share what test led you to this conclusion? Are you actually referring to this: "yes traffic from the another EC2 instance was blocked"? Because if that's the case, the traffic is blocked because once the vMX decrypts it, it sends it to the other EC2 instance unencrypted, so the ENI will see the ICMP packets.

If you mean traffic from AWS to on-prem is blocked when you block all outbound traffic on the vMX ENI, that is because if you do so (block all outbound), then the result is you are actually causing the tunnel to drop eventually (I say eventually because the tunnel may stay up for some time, but soon will drop. Or if you reboot the vMX you will see the tunnel won't come back up), because you are effectively blocking the vMX connectivity to the Meraki cloud and to your MX on-prem.

Let me know if your tests tell you otherwise.

[Stallone]

Hi @MarcoS 

It is not blocked by the Security Group because of the very nature of VPN. The vMX is a software that acts as the encryption termination point for your VPN tunnel, inside the EC2 instance; this means the ENI of the EC2 instance only sees encrypted traffic (in this case, Auto VPN traffic, which uses UDP encapsulation, choosing UDP ports in high ranges, 32768-61000 last I checked).

So, even when the security group has no inbound rules, effectively blocking all incoming traffic, it will not block the traffic, even if it is encrypted, because it is within a VPN tunnel that operates on the lower OSI level than the security group?

Can you share what test led you to this conclusion?

Sure, my setup consists of 2 AWS accounts, A and B, where account  A has a vMX ec2 instance with access to the on-premise resources.

A and B accounts can reach themselves via configured Transit Gateway, the goal of this infrastructure is to have access from AWS resources located in account B  to the on-premise resources behind the vMX instance.

So when I've added an inbound rule for the security group for the vMX instance that allows only ICMP traffic from my AWS account B subnet, I have only ICMP packets allowed to my on-premise subnet from the AWS 

I say eventually because the tunnel may stay up for some time, but soon will drop

Yes, you are correct, after some time all traffic wasn't able to reach on-premise resources.

[MarcoS · Accepted solution]

"So, even when the security group has no inbound rules, effectively blocking all incoming traffic, it will not block the traffic, even if it is encrypted, because it is within a VPN tunnel that operates on the lower OSI level than the security group?"

Having no inbound rules does not block your ICMP ping through the tunnel because your ICMP packets are inside the tunnel, the Security Group doesn't see them.

For your reference, here is how Meraki Auto VPN works: https://documentation.meraki.com/MX/Site-to-site_VPN/Meraki_Auto_VPN_-_Configuration_and_Troubleshoo... . We encapsulate packets in UDP, which simplifying means you have a packet (the traffic inside the tunnel) within another packet (the "outer layer"). This means that what the ENI (to which the Security Group is associated) sees is only the outer layer (which is what you see if you setup VPC Flow logs on that ENI) not what’s encrypted inside (your ICMP packets in your test).

Having no inbound rules does not cause the tunnel to drop because the Security Group is a stateful firewall.  Even if you block everything inbound, you can still establish the VPN tunnel because the connection is initiated from the inside of the EC2 instance and returning traffic allowed.

[Stallone]

Thank You @MarcoS for your support