Traffic not getting initiated from IKEv1 and IKEv2 for non-meraki tunnel for few selectors

[Tishman]

Hi Experts

I had created a site-to site tunnel with non-meraki device FTD with IKEv1 tunnel come up but for few traffic selectors traffic is not getting initiated from meraki but it works when initiated from FTD.

MX version 18.211.2

does anyone have any fix  as same is happening with IKEv2 when using FQDN.

[RaphaelL]

Hi 

Pretty sure I have the exact same issue that you have.

Rebooting the MX or up/down the tunnel seems to work for a couple hours then it stops working again. 

[Knowguy]

I have been fighting this exact same issue between an MX (both 18.107.10 and 18.211.2) and ASA (running latest suggested code, 9.18 interim).

Support has both suggested trying the patch 18.211.3 to see if it fixes the issue and also saying they don't believe that will fix the issue.

I have changed about everything that I can on the ASA side including disabling keepalives and NAT-T.

Adding 'route-lookup' to the end of the NAT exemption for the tunnel traffic

Adding and subsequently removing the IKEv1 command 'reverse-route'

Reordering ACL on ASA to match the order of private subnets on the Meraki.

Its seems sometimes random when one of the traffic selectors will fail.

But we can bring the specific failing selector up with this line in packet-tracer from the ASA side:

packet-tracer input inside tcp [INSIDE SOURCE IP] [DESTINATION IP] 443 detail

I have had two calls between TAC and Meraki Support and no solutions have been found.

Meraki supports last action was to disable multi-core support on the backend for the MX67 in my case since it has been known to cause issues on the lower end devices.

I thought this was going to fix the issue, it was stable for nearly 5 days and began to happen again... I think it was simply the reboot that fixed it which was necessary for the backend change.

I am actively trying to get this case escalated on both supports sides but having no luck. Went straight to the Cisco Rep too... nothing as of yet but I will let anyone know if I fix this.

I feel like it may have something todo with DPD / keepalives that at least one Meraki support engineer said on their end would be 'interval 10 retry 3' I belive the default on the ASA may be 'interval 10 retry 2'... But no telling what it is at this point.

[RaphaelL · Accepted solution]

Its seems sometimes random when one of the traffic selectors will fail.

But we can bring the specific failing selector up with this line in packet-tracer from the ASA side:

packet-tracer input inside tcp [INSIDE SOURCE IP] [DESTINATION IP] 443 detail

#### Exact same issue that we are experiencing. Nothing useful from Support up to now... 

[Knowguy]

Looks like support actually answered a similar post from Tishman here about upgrading to the patch I mentioned 18.211.3 but again nothing in release notes to indicate a fix for what we are seeing:

https://community.meraki.com/t5/Cloud-Security-SD-WAN-vMX/traffic-not-getting-initiated-from-IKEv1-f...

[RaphaelL]

18.211.3 hasn't fixed anything related to NMVPN for us. 

on a side note , really tired of the premade excuse : upgrade to the latest code and pray god that your bugs are no longer present.

[Knowguy]

Thank you for confirming that without me having another false hope fix! Hopefully we get some visibility from this discussion so please keep checking in if you learn anything new.

Not excited because I manage a few very similar environments that so far have avoided this….

Which doesn’t help when trying to troubleshoot to compare the other setups as I have already tested and reviewed all of the configuration differences to no avail.

[Tishman]

agreed but that is just work around each time we have to do that meraki should fix there issue

[Knowguy]

So again first thing this morning one of our traffic selectors to a single host IP was not coming up from initiating on the MX side.

I found this post for zScaler to MX and see them talking about recommended settings for Site-to-Site on the Meraki side:

Follow these recommendations:

• Security & SD-WAN -> Configure: Site-to-site VPN -> Non Meraki VPN settings:
• Preshared secret must be greater than 14 characters
• Authentication cannot be MD5
• Diffie-Hellman Group must be 14
• Phase 2 encryption cannot be NULL
• PFS can be configured to be either off or 14

The only thing we are doing wrong may be that we thought the pre-shared key needed to be under 14 characters, probably another forum resource post somewhere.

I am not sure why they are recommendations and where its coming from other than others experiences

I have scanned through the forum and have seen numerous posts now for similar issues that we are communicating about here:

https://community.meraki.com/t5/Security-SD-WAN/NON-MERAKI-Site-To-Site-VPN-network-translation-v18-...

https://community.meraki.com/t5/Security-SD-WAN/Non-Meraki-VPN-IKEv2-issues/td-p/245734/

https://community.meraki.com/t5/Security-SD-WAN/MX-to-Cisco-FTD-Site-to-Site-Using-IKEv2/td-p/245108

https://community.meraki.com/t5/Security-SD-WAN/Cannot-establish-VPN-to-non-Meraki-peer-Firepower/td...

https://community.meraki.com/t5/Cloud-Security-SD-WAN-vMX/traffic-not-getting-initiated-from-IKEv1-f...

https://community.meraki.com/t5/Security-SD-WAN/non-Meraki-VPN-peer-is-not-establishing-with-zScaler...

[RaphaelL]

We are already following these recommendations and still this morning one trafic selector is not working

[Knowguy]

So as I continue to watch the tunnel I see the VPN Registry: Partially connected warning. So I found another forum post here detailing what this means



https://community.meraki.com/t5/Security-SD-WAN/VPN-Registry-Partially-connected-What-does-this-mean...

Appears that you can find what ports your registry is using with the following information and a Meraki Support Engineer telling us about having them change the registry values to fix issues:

From JosRus | Meraki Employee

I would like to add some additional information to this:

When support initiates a change to your registry contact points, a migration period will occur, within which changes to your registry contact points cannot be performed. Any additional changes to these will require an additional waiting period while the migration finishes.

An additional port of 9351 has been added, which you will see is also listed under Help>Firewall Info>VPN registry. Upon issuing a registry IP change from our side, you will see the addresses on this page update automatically, so be sure to check this page after any registry IP change is made from the Meraki Support side, and update your upstream firewall/device rules with the new information accordingly.

[RaphaelL]

Good point but I think that this only applies to AutoVPN. This shouldn't affect NMVPN

[Knowguy]

Thought I would also throw out this other random mention of a bug that is not documented anywhere externally. Support is claiming there was a know issue with IKEv1 dropping random SAs:

Case 12086900
Tunnels should not have taken that long to reform. Could be running into a known issue with IKEv1 connections on 18.107.10+ and 18.209+ firmware where we can't reform specific child sa's.

Has anyone tested this theory by possible downgrading to any of the following?

• 18.107.8
• 18.170.9
• 18.208
• 18.208.01

All of these were considered "Stable" versions at some point.

[RaphaelL]

Is it specific to IKEv1 ?  I could ask Support to pin us to 18.208

[Knowguy]

That's what support said in my case but they never actually confirmed or denied this. Feel free to reference my ticket number. They were unsure if the lower end (MX67) in my case were stable on 18.208 or 18.107

[Knowguy]

So first thing this morning, All of the traffic selectors survived through the night and are still up and incrementing encaps/decaps.

[Knowguy]

Anyway we can mark this as not solved? We confirmed how to manually fix this but we need this to be properly addressed by Meraki.

[RaphaelL]

What is the manual fix ?

[RaphaelL]

Just had a tshoot session with support , they mentionned other customers with the same issue. Suggested to go to MX 19.1.3 which would contain an undocumented fix about that issue. Will comment if it works or not. Stay tuned.

[Knowguy]

You are very brave for doing this. But thank you. I will be watching. As for the manual fix above. I am just talking about being able to bring up the non-working traffic selectors using packet-tracer from the ASA side in my case.