Philip a good pointer. I tried adding a static to 8.8.8.8/32 via my next hop 192.168.2.1 via LAN 3 port and testing availability to host 8.8.8.8
I then added a 2nd static route to 0.0.0.0/0 via my next hop 192.168.2.1
I checked that the static route was active via the Fortinet Firewall 192.168.2.1 and it was.
I then failed the WAN on the Fortinet to fail the 8.8.8.8 host test and the default route did not change it remained active on the Fortinet.
Not giving up but using your logic, I added a 2nd availability test onto the 0.0.0.0/0 static also to ping host 8.8.8.8
This worked when the 8.8.8.8 host test was associated with the Default static. When I failed the WAN interface on the next hop firewall the route then disappeared.
Now for the interesting bit.
So we have a static default via a 3rd party firewall and have the ability to remove the route if the test ping to 8.8.8.8 fails.
My monitored pings to 8.34.34.34 and 88.221.170.233 both failed when the static default route disappears, however Internet connectivity is maintained by my test PC plugged into MX port 3 on a separate VLAN.
So the failover for Internet connectivity via the MX WAN port instead of the next hop worked 🙂 but the CMD pings stopped. I tired an additional CMD ping to a different destination 208.67.220.220 and that responded via the route MX WAN 1 however the original pings still showed Request Timed Out and again I still have internet connectivity.
So in my mind something is not timing out on the MX ARP around 8.34.34.34 / 88.221.170.233
I rebooted MX to see if pings would return and they did.
So were 99% there but I am a little confused what is happening re the original test pings failing. p.s If I re enable the WAN port on the Fortinet the Static route becomes active again and the pings return.
Strange. What do we think is going on here is it the arp cache on the MX that's not timing out within my observation window?
####################
Below is a test I have just performed pinging the 3 destinations. The original two that failed and the new destination to OpenDNS. The picture below is what just happened after I disconnected the Fortigate WAN. Notice the OPenDNS ping remains but the only the original destinations which were established when the Static route was active fail.
Got to be something in my mind to do with arp caches 🙂