Loop of route exchange between Hubs in DC-DC Failover

[MyHomeNWLab]

In the DC-DC Failover topology, routes configured in "Local Networks" loop between Hubs.
Therefore, it is necessary to contact support to "disable route exchange between hubs".

In my environment, it did not actually loop, but according to support, looping is an expected behavior.

I had it investigated, but the cause was unknown.

I have asked Japan Support to describe this in the documentation.
However, Japan Support does not have the privilege to edit the English documentation.

As an alternative, I asked them to describe it in the Japanese [Localized] documentation.
Please specify in the English documentation as well.

[Japanese Doc]
データセンター冗長化(DC-DC Failover) 設定ガイド - Cisco Meraki
https://documentation.meraki.com/MX/Deployment_Guides/Datacenter_Redundancy_(DC-DC_Failover)_Deploym...

> ネットワーク環境によってはハブ間通信でループが発生する場合があります。そのような場合はハブ間での経路交換を停止することで回避できる可能性があります。ご希望の方は Cisco Meraki テクニカルサポートまでご相談ください。

Translation into English.
Loops may occur in inter-hub communication depending on the network environment. In such cases, it may be possible to avoid the problem by stopping the route exchange between hubs. Please contact Cisco Meraki Technical Support for assistance.

[MyHomeNWLab]

The difference between expected and actual behavior has a significant impact on network design.

If not documented, many environments are unknowingly affected.
In fact, there appears to be no mention of it at all in Meraki's English documentation.

I learned about it from the following Japanese article. (It is not official for Meraki.)

https://apicli.com/2019/04/20/meraki-mx-%E3%82%92-dc-%E5%86%97%E9%95%B7%E7%92%B0%E5%A2%83%E3%81%A7%E...

And no loops occurred in my personal environment or in the production environment at work.
So it is possible that in many DC-DC Failover topologies, the route exchange between hubs is not disabled.

[Ryan_Miles]

There's more detail here on what will create a loop. Simply having two hubs advertising the same local network shouldn't cause a loop. If however you use OSPF on the hubs to peer with DC routers and then those DC routers also peer to each other you could introduce a loop. In these scenarios you can do route filtering on the routers to prevent loops/ensure expected routing paths. Also, Meraki Support can disable hub to hub tunnels if required due to the topology design.

Each network is different and any redundant DC designs should be reviewed with your Meraki SE to ensure it will work as expected. 

[MyHomeNWLab]

@Ryan_Miles
Thank you for your reply.

Just to be sure, I will add some background to my contact with support.

I first learned of loop behavior from an informal blog by a Cisco employee (Japanese contents).
Due to unofficial information, I have confirmed the specifications with support.
On top of that, the actual behavior was different, so I asked support to investigate.
For this reason, I have contacted support twice to be sure.

> If however you use OSPF on the hubs to peer with DC routers and then those DC routers also peer to each other you could introduce a loop. In these scenarios you can do route filtering on the routers to prevent loops/ensure expected routing paths. Also, Meraki Support can disable hub to hub tunnels if required due to the topology design.

Thanks for the very good information.

Yes, I think there are cases of loops caused by DC routers.

That interpretation would explain why it is not looping in my environment.
However, the display on the Dashboard appears to be looping regardless of other devices (e.g. DC routers).

Looking at the Route table in the dashboard, the Local Networks route appears to be looping.

In contrast, with "disable route exchange between hubs," the routes do not appear to be looping.

I am not sure if the Route table for Meraki MX is FIB or RIB...

Will this just be a display issue on the Dashboard?

I am not sure which information is correct.

[MyHomeNWLab]

If you say it depends on topology, I would like to change my view.

Was "disabling the Hub-to-Hub route exchange" not necessary in the configuration described in the document?

In other words, isn't a loop an absolute behavior?

Datacenter Redundancy (DC-DC Failover) Deployment Guide - Cisco Meraki
https://documentation.meraki.com/MX/Deployment_Guides/Datacenter_Redundancy_(DC-DC_Failover)_Deploym...

[MyHomeNWLab]

Without disabling the Hub-to-Hub route exchange, it appears to be looping on the Dashboard.

[Settings]

* MX#1 side

* MX#2 side

[Route table]

* MX#1 side

* MX#2 side

[Ryan_Miles]

The hub route tables look normal given how hub to hub tunnels work. That doesn't indicate there's a loop.

Route prioritization handles how duplicate routes are used. 

Is spoke traffic working to the destinations?

[MyHomeNWLab]

Thanks for your kind reply.

> The hub route tables look normal given how hub to hub tunnels work. That doesn't indicate there's a loop.

Thank you for confirming.

I interpreted that if the tunnel was functioning, the route was also being learned.

> Route prioritization handles how duplicate routes are used.

Yes, I understand. That document (Route prioritization) has been checked.
However, I am confused as to how to interpret the Route table.
It does not appear to be displaying the FIB like the "show ip route" command in Cisco IOS.

I have already checked the following a document.
I am not sure exactly how to look at the Route table, as the detailed behavior was not described.

Route Table - Cisco Meraki
https://documentation.meraki.com/MX/Networks_and_Routing/Route_Table

> Is spoke traffic working to the destinations?

Yes, it is working. Traffic from the spokes to the hub can communicate.
There is no indication that traffic is looping.

However, when I contacted support, I was told that looping is an expected behavior.
I'm not even told that it loops under any particular conditions.

It contradicts what we are conversing about on this topic.

I want to resolve the discrepancy somehow.