Route tracking issue-WHat could be the issue for this.

[Merakibud]

Oct 17 10:14:51 Route tracking Route connection change peer_type: gateway, peer: 192.168.101.9, connection_status: connected
Oct 17 10:13:54 Route tracking Route connection change peer_type: gateway, peer: 192.168.101.9, connection_status: disconnected
Oct 17 08:53:10 Route tracking Route connection change peer_type: gateway, peer: 192.168.101.9, connection_status: connected
Oct 17 08:52:39 Route tracking Route connection change peer_type: gateway, peer: 192.168.101.9, connection_status: disconnected
Oct 17 08:11:58 Route tracking Route connection change peer_type: gateway, peer: 192.168.101.9, connection_status: connected
Oct 17 08:11:34 Route tracking Route connection change peer_type: gateway, peer: 192.168.101.9, connection_status: disconnected

I am having Issue with the loosing connectivity to the FortiGate firewall as seen above. In my network All the branch traffic comes to this site and Internet traffic is forwarded though the static route to the FortiGate and to the  Internet through ISP gateway. what could be the issue for loosing connectivity.

[RaphaelL]

Is the Fortinet dropping ICMP packets ? The MX will send ICMP packets all the time and if the destination stops responding , it will give a log like that. 

If you can , run a packet capture and try to corelate the logs and what you observe in the packet capture. That might help.

[Merakibud]

During the issue happening Fortinet is reachable but the Meraki Firewall is unreachable and no internet connectivity.

[RaphaelL]

Any logs on the Fortinet that could help ? Have you tried opening a ticket with Support ?

[Merakibud]

yes getting logs from FortiGate. Have opened the ticket with meraki

[Merakibud]

During the issue happening Fortinet is reachable but the Meraki Firewall is unreachable and no internet connectivity.

[cmr]

Why do you have the Meraki on the internet, yet send the traffic to the Fortinet?  Why not just have the Meraki behind the Fortinet?

[Merakibud]

client want Fortinet behind the Meraki firewall.

[cmr]

So why is the Fortinet directly connected to the Internet as well, VPN?

[Merakibud]

Meraki as providing the site to site vpn.

[cmr]

So why not just have:

LAN -> Fortinet -> Meraki -> Internet

[Merakibud]

We needed all the branch traffic come to the Meraki and then to the Fortinet and to the internet.

we could have Meraki as VPN concentrator but since we have Meraki in routed mode we placed the fortinet behind the MEraki

[DavNel]

its uncanny but we are having what seems to be the exact same issue with our Meraki and Fortigate.  Below is our setup, but essentially we see the same sort of messages in our MX95, it looses connection to our Fortigate and as a result all our spoke sites go down and headoffice looses internet connectivity. 

Been going on since Oct 12th, with about 5 flaps a day lasting about 30 seconds each, then it comes up on its own.

when it occurs fortigate is unable to ping mx95 but the fortigate itself still has internet access.

laptop plugged directly into a lan port on the MX95 is not able to ping the mx95 or 8.8.8.8 when the outage occurs but is able to ping the fortigate

Have you solved your problem yet?

[DavNel]

It would seem out issue has been isolated to the Cisco Merake IDS (Intrusion Detection System).  The IDS is currently disabled and we haven’t had an outage since.

[BNITM]

Hello,

could you confirm that disabling IDS solved this issue? We have a similar problem using MX95 with a couple of network outages a day (approx. 20 / week) showing the same route connection change messages that includes all of our other routers within the LAN, but typically not the uplink (which is not a Fortigate device but a Unifi Edgerouter). Generally, our topology is quite different using a warm spare and a MS225-48LP as root switch for approx. 50 Unifi switches, but I am desperately looking for any clues to this problem.

[DavNel]

Yes disabling the IDS fixed the issue.  While it wasnt exactly the IDS that was the issue it was the most recent definition package for the IDS.  But disabling the IDS also then prevented the meraki from using that bad definition package.  There was likely something in it that was either bad or causing a false positive.  

[MartinLL]

I cant remember which version of MX this occured on, but the snort engine was upgraded from SNORT2 to SNORT3 at some point. On MX95 and MX85 this is a known issue and you see the route tracking drops due to the snort engine crashing. When SNORT restarts the dataplane stops forwarding traffic and the outage lasts for about 1 minute.

You have two options. Disable IDS or contact support. They can manually downgrade your network from SNORT3 to SNORT2.

[BNITM]

Hello, thanks for your replies. This is really interesting, especially @MartinLL 's SNORT investigation. After disabling IDS I had no route change events and network outages since 10 days. This is the longest period of time so far since we started working on the problem. Hopping that we got rid of it, I think.I can do without IDS.