MGCP traffic issues through an MX

[StevenRandolph]

Has any one experienced Meraki MX devices that decide to stop sending MGCP packets. I have a network of 65+ devices setup in a Hub spoke design with multiply VPN concetrators that has operated for over 2 years. Now in the last 3 weeks all of a sudden spoke branches can not make calls to other spokes. However a hub to spoke works and a hub to hub works. Packet captures show the MGCP packets between hub and spokes, but when trying the spoke to spoke capture we never see the MGCP packets transmit. Of course the Meraki support wants to say its the border switch or ASA, However we have not made any changes to those devices in the last 2 or 3 months. The only real changes that have been made was the Firmware update of the Meraki MXs back in October.

[PhilipDAth]

If all the traffic is flowing over VPN then it should not matter what type of traffic it is.

Do you have more than one hub offering a destination to the MGCP server?  And if so, is their a chance you have managed to develop asymmetric routing?

[StevenRandolph]

Only one hub servers for the primary server location so there is no possible way to have asymmetric routing. Presently i have 2 hubs one located in the US and one in EU. i have a third Hub that stays offline unless we need to turn up the US DR site.

[PhilipDAth]

So with spoke to spoke communications the traffic will go via a hub (unless all sites are configured as hubs).

Do some quick packet captures to determine:

1. Starting at a spoke, do you see the MGCP traffic entering the MX via the LAN?
2. At the hub, do you see the MGCP traffic arriving via VPN?
3. At the hub, do you see the MGCP traffic leaving via VPN?
4. At the recipient spoke, do you see the MGCP traffic being transmitted on the LAN?

[StevenRandolph]

My voice guys has been battling this over the 2 weeks, initaily it was 2 or three sites and now it is network wide. Initially we thought is was a Shoretel issue. it wasn't until Thursday last week they brought me in and this is what i have found when performing packet captures.

Captures from Hub to Spoke or Spoke to Hub depending on the who initiates the call

1. Captures show the initial MGCP packets and all following UDP packets

2. calls complete as normal

Captures of spoke to spoke

1. it dose not matter who initiates the call we never see the initail MGCP traffic its like it is getting dropped before it leaves the LAN of the initiator. However we see the UDP packet from the initiator but the call fails due to the fact the intial MGCP are dropped.

well that sounds like a VOIP switch issue but  if you go for hub to spoke or spoke to hub it works. Well i  thought let configure 2 spokes that would not work, as hubs. If it is being block at the primary hub the call should not work right, well here is the confusing part. after i configured the 2 spoke site as hubs the call went through between them. Keep in mind that all MGCP packets still have to transverse the primary hub in order to set up the call.

 So i stepped back and took a capture of the calls and low and behold i see the MGCP packets transverse the whole network.

Once i place them back into spoke mode we don't even see the MGCP packets on the LAN all over again.

[PhilipDAth]

This piece is 100% crystal clear - if you do a packet capture on the LAN interface of a spoke calling another spoke and see nothing - then no traffic made it to the MX - at all.

If no traffic is making it to the MX - then it can not be an MX issue.

[StevenRandolph]

The problem is it is present when the MX is in hub mode, however when i change it back to spoke the only thing that isn't there is the MGCP packets. so if isn't a meraki issue what is it. because it isn't a VOIP switch issue. Keep in mind all other traffic works just fine.

[PhilipDAth]

So you are saying if you take your existing hub, change it to be a spoke (so everything goes through your one remaining hub in another country) that it starts working?

[StevenRandolph]

Well as a blind hail Mary pass, while I have been on this form I have had my hands on tech in Cali perform a factory restore on the primary US hub, and low and behold all of my US site can now call with no problem. Spoke to Spoke, Hub to spoke. Looks like I will leave my EU hub alone until I can get a Meraki engineer online to see this s**t.

[PhilipDAth]

Something very messed up happened.  What software version are you using at the moment?

[StevenRandolph]

12.26

[PhilipDAth]

I have had lots of niggles with the 12.x versions.  Personally I would move to 13.28.  It works flawlessly.