cancel
Showing results for 
Show  only  | Search instead for 
Did you mean: 

Spurious Uplink failover messages from both MX appliances - ISP says everything's fine on their end

SOLVED
Conversationalist

Spurious Uplink failover messages from both MX appliances - ISP says everything's fine on their end

For about a month now we've been getting periodic uplink failover alerts originating from both our primary appliance and the spare. They're at random times during the day, mostly overnight between midnight and 5 AM. The office is empty during those times so traffic is basically at nothing. A few weeks ago the alerts were infrequent enough to make us believe that it was just a result of ISP maintenance after hours. However, lately it's been frequent and happening throughout the work day. But here's what's weird, nothing actually fails over. There no packet loss, nada and the the failover lasts a minute or less. Every time it's like this. End users at the site don't notice any issues as well, so I don't believe anything is failing over. Here's the troubleshooting we've done so far:

 

  • Called ISP multiple times to determine that there's no issue on their end. They say the circuit has been up and stable for over 3 months.
  • On our initial call with Meraki, it was determined that our MX firmware was out-of-date and so we upgraded to the latest stable version. We saw no problem for about 20 hours and then it started happening again.
  • Second call with Meraki and they determined that the checks were failing at the Meraki cloud portion on UDP port 7351. However, we're not blocking any outgoing traffic in our firewall settings. Could I be missing something here?

I'm not sure where to go from here. I don't think anything is failing over, I think the alerting on the Meraki end might be a little too sensitive and it's generating these alerts.

 

Here's a sample of the failover messages:

 

The primary security appliance in the <CorpNetwork>  network switched to using its primary uplink, configured to be uplink Internet 1, after a period in which the link was unavailable.

There have been a total of 2 failover events detected:

At 05:07 AM EST on Nov 4, the security appliance switched to using Internet 2 as its uplink.
At 05:07 AM EST on Nov 4, the security appliance switched to using Internet 1 as its uplink.

Of course any insight into this would be very much appreciated.

1 ACCEPTED SOLUTION

Accepted Solutions
Kind of a big deal

Re: Spurious Uplink failover messages from both MX appliances - ISP says everything's fine on their

In my experience, yes. Primary uplink failure is extremely sensitive. We have a client that often fluctuates, and we had to disable that alert on them because it was tons of noise. The client has never experienced noticeable side effects.

6 REPLIES 6
Kind of a big deal

Re: Spurious Uplink failover messages from both MX appliances - ISP says everything's fine on their

In my experience, yes. Primary uplink failure is extremely sensitive. We have a client that often fluctuates, and we had to disable that alert on them because it was tons of noise. The client has never experienced noticeable side effects.

Conversationalist

Re: Spurious Uplink failover messages from both MX appliances - ISP says everything's fine on their

What happens if the link is really down? How do you receive alerts?

Kind of a big deal

Re: Spurious Uplink failover messages from both MX appliances - ISP says everything's fine on their

For this really noisy client, we've accepted the risk - but we can do that with them.

 

Otherwise, we're working on alerts using a third-party app called Auvik.

Conversationalist

Re: Spurious Uplink failover messages from both MX appliances - ISP says everything's fine on their

Has Meraki confirmed that this is an issue? 

Highlighted
Kind of a big deal

Re: Spurious Uplink failover messages from both MX appliances - ISP says everything's fine on their

Might be a good question for you to ask support.

cmr
A model citizen

Re: Spurious Uplink failover messages from both MX appliances - ISP says everything's fine on their

We get the same on one (busy) site, so much that I made the other ISP the primary.  The issue still persists, though perhaps a little less frequently.  Having seen this I'll log it with support as both ISPs cannot see an issue and the users don't either.

 

Funniest thing is that they are both MPLS links and the default routes are on the same stack at our main DC and from there they both go through the same internet firewall, even on the same rule, so not sure how one is down when the other is up...!

Welcome to the Meraki Community!
To start contributing, simply sign in with your Cisco account. If you don't yet have a Cisco account, you can sign up.