You need to use Virtual IP (VIP).
WAN Virtual IPs
VIP addresses are shared by both the primary and warm spare appliance. Inbound and outbound traffic use this address to maintain the same IP address during a failover and reduce disruption. The virtual IPs are configured on the Security & SD-WAN > Monitor > Appliance status page, under the Spare section in the upper-left corner of the page. If two uplinks are configured, a VIP can be configured for each uplink. Each VIP must be in the same subnet as the IP addresses of both appliances for the uplink it is configured for, and it must be unique. In particular, it cannot be the same as either the primary or the warm spare's IP address.
https://documentation.meraki.com/MX/Deployment_Guides/MX_Warm_Spare_-_High_Availability_Pair#WAN_Vir...
I am not a Cisco Meraki employee. My suggestions are based on documentation of Meraki best practices and day-to-day experience.
Please, if this post was useful, leave your kudos and mark it as solved.