Microsoft Teams - L7 country blocking acted up a bit weird today (did it do so for anyone else) ?

[thomasthomsen]

We had a fun problem today.

This morning Microsoft teams acted up very strange, seemed online, but we could not send messages, or call anyone.

No update of the MXs in the environment had been done, and they are running different releases.

Pinging teams.microsoft.com you got no reply, and a packet-capture on the LAN side of the firewall said that all SYN's toward that resolved IP (52.123.128.14) was only replied with RST.

Capturing packets on the WAN side towards 52.123.128.14, nothing, so it was pretty clear that the MX was the one "eating" these packets.

To begin with I suspected content filtering, because the site we started to troubleshoot was running an old MX software 18.1xx.x, and I have some experience here where content filtering was doing "something wrong(tm)".

But we then had reports of other sites, where luckily they where running a 18.2xx.x, and here the firewall log feature told us that traffic was being blocked by L7 rules.

The only L7 rules the customer uses are country filtering, but the same list on all sites. Removing the list, problem solved.

We are now trying to figure out what this IP has now been "assigned to". It should be US, that is not blocked in the list, but clearly something has happened.

I dont even know who maintains the country list on MX ... Talos ?

/Thomas

[thomasthomsen]

The country that it was block on was : "Qatar"

[NordOps]

Thanks!

[MannyElPollo]

Hey Thomas,

Certainly- had some of our clients that are OPEN on Sunday's experience this-

COUNTRY = QATAR

We opened a meraki case "12051425" which per ticket notes a request has been made to MaxMind to fix the country flagging (MaxMind is used by Meraki GeoIP)

[thomasthomsen]

Thanks for the info, that was one of the providers I did not know Meraki was using.

[thomasthomsen]

PS: I immediately wish for the Firewall log feature to tell me WHAT L7 rule / country (in this instance) the block is coming from 🙂

[CWHLOUKY]

Oh to dream...  That and fixing it so we can choose the order a device is named to fix the mDNS naming issue they created a couple of years ago, 

[MannyElPollo]

Hehehe- COMPLETELY AGREE- still shocked L7 firewall events do NOT get logged 😕

One thing you can do (after firmware v18+ if recall correctly) is use the TOOLS > FIREWALL 

And you can filter a host to see what gets blocked/etc. sometimes helps 🙂

Thanks!

[PhilipDAth]

Sometimes this can be caused by ISP routing.  The normal path to teams.microsoft.com is not available (for example), and it gets routed to the next closest, which is in another country.

Sometimes you get actual attacks, like BGP Hijacking, to divert traffic via another country.

https://news.sophos.com/en-us/2018/10/30/china-hijacking-internet-traffic-using-bgp-claim-researcher...

It is not clear from the information wether this block was correct or not, because you don't mention in what country your access to teams.microsoft.com was being served by.  You would have needed to do a minimum of a traceroute at the time to check.

[Malwina · Accepted solution]

Hi All! Maxmind has released a new list with an update for the two Microsoft ranges.

MXs should now have the new list with the corrections. 

[dlevasseur]

Confirmed.  I didn't apply the changes because nobody cares about Teams except those that send us Teams meetings.  Today all is good.

[SwissAmi]

Hello Thomasthomsen,

Just to confirm: What a strange coincidence! I experienced the exact same behavior and the problem was that Meraki (or Maxmind) was detecting the IP addresses for Microsoft Teams in Qatar.

Greetings

[Keristopa]

Is there a way for us to replace the current IP geolocation database with IP2Location, which is known for its higher accuracy?