Loss utility shows 100% for 1.1.1.1 but is normal for 4.2.2.2 and 8.8.8.8

[McDouble]

Hi community,

We have a customer with hundreds of sites and we constantly get tickets due to one of their helpdesk techs seeing high loss rates out 1.1.1.1. It's important to note that they have multiple paths for redundancy, essentially 2 routers and 2 switches that are connected to each other twice with RSTP blocking enabled. I have attached their topology. 

I have two questions:

#1 Why does 1.1.1.1 show 100% loss at so many of the sites so often

#2 Why does the utility show WAN1 and WAN2 3 times?

This is my suspected reason for having WAN1 and WAN2 listed 3 times- 

For each WAN, respectively (flipflop the 1s and 2s for the 3 WAN2 connections):

SEC1 to SW1 to ISP1

SEC1 to SW2 to SEC2 to SW1 to ISP1 - twice: Two ports connect between each sw to sec, 

I have NO idea why we constantly see the 100% loss rates for 1.1.1.1. It causes our customer's help desk to generate tickets that the WAN is down because they see the loss rates in the graph. It drives both our NOC and the customer's NOC crazy. 

Topology: 

Any ideas on causes, fixes, workarounds?

[PhilipDAth]

I don't know what part of the world you are in, but the CloudFlare 1.1.1.1 server does not perform well in my part of the world.  Of the public DNS services, it would be the worst performing.

So when I here you are getting connectivity loss to it - I think it is probably whatever 1.1.1.1 node you are talking to, rather than your connections.

[BlakeRichardson]

Agreed, after all the hype about CloudFlare and how quick it was compared to Googles DNS I have also found it performs poorly. I used it for a day before switching back to Google. 

[JohnM]

You should try using it with non-Cisco gear, 1.1.1.1 works extremely well and resolves super fast... except with Cisco/Meraki. Why? Because Cisco has, over the years, improperly treated 1.x.x.x. as fair game for their own uses.

https://blog.cloudflare.com/fixing-reachability-to-1-1-1-1-globally/

Who else has been mis-using 1.1.1.1 more than others?

Using the RIPE Atlas probes gives excellent visibility into residential and business internet connections, however they’re connected via a cable, so this rules out another use-case, WiFi access points. After very little research we quickly came across Cisco mis-using 1.1.1.1, a quick search for “cisco 1.1.1.1” brought up numerous articles where Cisco are squatting on 1.1.1.1 for their Wireless LAN Controllers (WLC). It’s unclear if Cisco officially regards 1.0.0.0/8 as bogon space, but there are lots of examples that can be found on their community websites giving example bogon lists that include the /8. It mostly seems to be used for captive portal when authenticating to the wireless access point, often found in hotels, cafés and other public WiFi hotspot locations.

---

[PhilipDAth]

>You should try using it with non-Cisco gear, 1.1.1.1 works extremely well

What you meant to say was - it works really well for whatever bit of the world you are in.  That simply isn't the experience for some of us.

Cisco don't regard 1.0.0.0/8 as bogun space.  Cisco did make a tremendous mistake using that as an example IP address in their documentation when configuring guest access on the WLC's - which everyone then cloned.  I can't believe they did that.

No other Cisco kit gives special treatment to 1.1.1.1.

[Schmitty]

Actually depending on where these clients are located and if they are behind say an access point and you have any type of splash page configured your Meraki device advertises it's splash page as 1.1.1.1

Maybe this will help maybe it will not. You can tell if this is happening if you run a pcap you will see ARP from the Meraki device for 1.1.1.1