Juniper Mist Switches cannot connect to Cloud behind Meraki MX 67 firmware 19.1.3

[CharlieCrackle]

I have been debugging a problem for a week in the evenings with Juniper Mist Switches not connecting to cloud when behind Meraki MX 67.  19.1.3

Note: Mist Access Points are connecting Fine.

From the hardware Point of view they look connected (IE CLD Light solid white)  and the connection to 3.105.198.133:2200 shows  ESTABLISHED

Just he Cloud console show them disconnected

If the MIST switches are connected to a MG51 with a 5G SIM card.    The Mist Switches work

Using the Internet connection  with a TPLink Home Router BE9300…   The Mist Switches work 

I added a Test MX 64 to a new network (test MX64) and using same internet link  Firmware 18.107.10  The Mist Switches work

Downgraded the MX67 to 18.211.2  and the Mist Switches work !

So must be something in 19.1.3 

Looking at the packet capture  the Meraki does something to the packet at the  "SSH-2.0 Go"   part of the conversation  and the MIST server RESETS the TCP session.

They have a chat and  switch sends

MSG-ID: MXOC-DEVICE-NOTIFY

MSG-VER: V1

DEVICE-ID: <snip>

MXOC-TOKEN:

MXOC-TIMESTAMP: 1721596480.2041769

Then

MSG-ID: DEVICE-CONN-INFO

MSG-VER: V1

DEVICE-ID: <snip>

HOST-KEY: ssh-rsa <snip>

HMAC: <snip>

SSH-2.0-Go

And then the TCP  RESET from the mist end.    the Meraki MX has done something to the packet the server does not like...

Case 11978964 (for all the packet captures)

[jbright]

MX 19.1.3 is an early beta firmware. I have had to recently migrate several customers that were early adopters back to 18.211.2 due to several different problems, so the problem you experienced likely occurred due to bugs in it too.

[CharlieCrackle]

Interesting update....   I have got it working now on 19.1.3  but only when threat Protection Turned off.    I have  AMP enabled and IDS set to Prevention and Security

After some analysis   The traffic is being hit with   

SID 1:58992      MALWARE-CNC -- Snort has detected a Comand and Control (CNC) rule violation, most likely for commands and calls for files or other stages from the control server.

Which all make sense.  Mist Cloud is a command and control server.    It seems Cisco do not whitelist their competition products as part of "business critical applications"

I am also seeing  

SID 128:3  SSH version string is greater than the configured maximum.

No idea how you configure this one...

[cmr]

'It seems Cisco do not whitelist their competition products as part of "business critical applications"' - this is a feature 😉

[CharlieCrackle]

I have tried to white list Mist (and working so far) but Mist can change the IP at any time. so I know it will bring down the network in future   Need to be able to enter as DNS names not IP.

Merkai can we please get All Mist communications added to "business critical applications"' Please.

[CharlieCrackle · Accepted solution]

Just an update on this topic for all you MIST users...  Meraki support has come back that it is by design..

"Mist Cloud is a command and control server so snort is doing what its expected of it"

Interesting "Meraki Cloud is a command and control server too and not effected 🙂

I have been told I need to open a separate support case with the NBAR team  and it will take a year or so to implement !!!

Go figure how you even go about raising this case.  Once implemented then meraki can use NBAR to add mist to " Business Critical Application " list.

[cmr]

I'd open a case ASAP and I'm sure it won't take that long as it would be anti competitive.