We bought 2X MX100 Security Appliance (retail price at $4999 each + License ). Currently running at the latest Stable firmware 12.24 and It blocks all device from downloading windows update and Adobe update even thou I whitelist all known Microsoft update sites. Meraki solution
1) Disable Amp ( Risk of getting Malware )
2) Upgrade firmware to V14 BETA. ( Running critical production network on BETA Firmware? )
Anyone have better workaround please help !
Solved! Go to solution.
I am not sure what it was removed, there was nothing in there that was a privacy concern. Anyway, earlier I was testing with a Win 7 box, when I tested with a Win 10 box, bam right away Windows Update broke. I am running MX 12.24 on this MX 100, I moved the client over to my MX 250 running MX 14.XX and right away the updates started working. I can confirm there is an issue here and I was able to replicate it exactly as you described.
Ryan
I've deployed a lot of MX's - and they have never blocked Windows Updates without being configured to do so.
Have you configured are layer 7 firewall rules? Can you configured any content filtering rules?
Hi PhilipDath,
L7 only block All P2P, Video and Music and Gaming.
Content Filtering only blocks some category that has nothing to do with Microsoft update and Adobe update. ( Unless Microsoft uses P2P Protocol to push update? )
This only happens on my MX100. I have many MX65W with the same config without any issue.
I called Meraki support twice regarding this issue for a month now and they gve me the same answer.
Hope to hear back from some other MX100 users.
Try removing the L7 rules and see if that fixes it. If not put them back. Repeat with the contenting filtering rules.
One of those items should get it working again. Tell us which one it was.
Hello @Jack
I have an MX100 sitting as a cold spare to our MX250. I will fire this up and create a test network and try to duplicate the issue. When we had the MX100 in operation AMP was grabbing Console8 updates as malicious. I am assuming AMP is enabled and what are you IDS settings? Prevention and Balanced? Just want to duplicate your settings here.
Thanks for your help. IDS set to Prevention - balance.
Hello Again @Jack
I have a spare MX100 running 12.24 that I reset back to factory and I enabled AMP and IDS like you have, see screenshots. I also added the L7 rules you mentioned above. I happen to have an extra connection to the outside world with a public IP, so there is not a double NAT taking place here. I had no problem fetching updates from windows update servers or adobe updates. if this traffic was getting grabbed by IDS or by AMP, there would be a log of that event that is easy to find the in security center.
This very much sound like an issue with Content Filtering, more specifically IP/URL reputation as @PhilipDAth mentioned.
"In firmware version 13.3, URL reputation was prioritized over IP reputation, as opposed to IP reputation being the deciding factor on previous firmware versions. If, for some reason, the IP has a different categorization then the URL, the client could be allowed through."
I can tell you that I am running MX 14.15 on an MX250 and I have not been adversely affected by this beta firmware in a production environment with 1000+ daily clients.
"If a client is being blocked from accessing a page, the easiest way to tell whether content filtering is blocking the traffic is to check your Event Log. When looking at the Security Appliance's network in the dashboard, navigate to Network-wide > Monitor > Event log. To help narrow down the scope, the event type 'Content filtering blocked URL' can be included in the 'Event type include' field."
I hope this helps.
Ryan
If you look in the event log for the network - what is the exact reason it gives for the blocking the traffic?
I dont think Event log shows whats being block on AMP. Any Idea what event to sort?
If you go:
Security Appliance/Security Centre/Events
Does anything come up?
And you are saying that if you disable AMP it starts working? If there is nothing in that log then it should mean that AMP is not blocking your traffic.
The beta firmware is pretty good. You are unlikely to find any issues if you upgrade to it.
We can probably solve this now we know the IDS is triggering.
Go:
Security Appliance/Threat Protection/Intrusion detection and prevention
Under "Whitelisted Rules" click "Whitelist an IDS rule". Select the rule that is firing above the in the log.
@Jack May I get a screen capture of your content filtering and layer 3 / 7 rules?
You are running MX 12.24 correct?
Ryan
I think you might be affected by the IP Reputation/URL filtering issue. This was resolved in 13.3. I think you should upgrade to the beta firmware.
You can read about the issue here:
"Sometimes, sites will be blocked even though their URL category is not blocked. Usually this happens when the IP has a bad reputation but the URL reputation is good. This happens commonly with very large domains like Google that own many IP addresses and sometimes purchase new IP addresses that have not yet been re-categorized to take their new owner into consideration. In situations like this, these IPs sometimes have a category of 'Phishing and Other Frauds,' or various other categories that may actually be blocked:"
Meraki support told me that V13 will not even solve my issue. I have to schedule a firmware update and they need to manually push V14 for this issue to be resolve. But its on Bata. Scary. I just dont understand why Cisco Meraki cannot make a windows update to work on a stable firmware?
I would assign a group policy only to the server to disable AMP just for those devices. Then try windows update again.
Seen similar problem with MX64/65/84.
Found that it corrected by turning AMP off, waiting a bit (minutes) then turning it back on, this allowed updates to proceed.
Havent seen the problem in a while, so may have been covered in a recent update - we are running typically newer than stable release.
@Jack I wrote out a really lengthy reply and added screenshots, it now disappeared or was removed, did you get a chance to see that reply?
I saw it on my email and Im trying to reply and then its gone on the forum .. someone deleted it maybe for privacy issue? Its funny that it works for you but not me. I did not have the chance to look at your screenshot. I guess i have no choice but to upgrade to the new beta firmware... im sure it will work. Worst case Revert back to V12. Thank you again for all your help !!
I am not sure what it was removed, there was nothing in there that was a privacy concern. Anyway, earlier I was testing with a Win 7 box, when I tested with a Win 10 box, bam right away Windows Update broke. I am running MX 12.24 on this MX 100, I moved the client over to my MX 250 running MX 14.XX and right away the updates started working. I can confirm there is an issue here and I was able to replicate it exactly as you described.
Ryan
I understand the frustration, however I think it might be OK now to upgrade to 14.X if your willing. That seems to have fixed the issue based on others from this thread.
This will be the third *major* issue that we've encountered this year where the fix was installing beta firmware. That's nuts.
Disabling AMP for 10 min and enable it works for me. Try that.
Hello @CarolineS,
Thank you for jumping in here and letting us know, so nice to have some Cisco Meraki presence here.
I'v been having same issue since 2016. A couple of our Meraki sites (MX64's) have reported file download failures when AMP is enabled. This issue manifest itself in a weird way, they work sometimes.
This is a known issue with Cisco Meraki AMP, Sometimes files will change disposition based on new threat intelligence gained by the AMP cloud and sees clean files as Malicious, then blocked.
Per Meraki, most customers are experiencing similar issues and they are working on a permanent fix soon????
Since we don’t want to disabled AMP as a fix, here is a workaround;
Yup seen this too. Meraki MX64 and 64W.
Solution is to add site to whitelist, turn off AMP - wait, turn on AMP - wait.
I whitelisted the following for Windows Updates..
microsoft.com
windowsupdate.com
Meraki filtering assumes all subdomains allowed as well on the above.
Thank you, just wanted to reply stating c0sm0's workaround fixes the issue. I'm running WSUS for domain joined machines, but some BYOD laptops on our wifi could not get windows updates over the internet. Running MX100 and MX64 on 13.3
I just had a similar issue, and wanted to describe it for others' reference. All Windows 8 era machines (8, 8.1, WS2012, WS2012 R2) would not update and gave the error code 0x8024402F. This began seemingly sporadically in November of 2018, and audit logs did not show any system configuration changes around that time. The ultimate cause appeared to be AMP blocking Windows from downloading legitimate .cab files from Microsoft websites. In the Security Center event logs, no events were posted indicating that any blocking had occurred. After searching for other issues with our client machines or content filters, we were able to solve the problem very simply by merely disabling AMP and re-enabling it shortly thereafter. The updates started flowing again just fine after resetting AMP in this way, and we have not had any issues with downloading legitimate .cab or .diagcab files since. I'm not sure if there was some hang in the process that is supposed to be scanning .cab files or with the malware definitions in AMP, but toggling the enable configuration fixed the problem.
For this problem, the best solution is to install the 14.x beta firmware. It prevents AMP from getting "indigestion" and blocking downloads based on false positives.
My procedure is:
Restart AMP: disable it, save, wait for MX to update its config, then re-enable it.
Enable beta firmware under Network wide->General and schedule an update for maintenance window via Organization->Firmware Updates.