With changing the static IP I mean going to Security & SD-WAN > Appliance Status, then going to the Uplink tab, and changing your IP there on beforehand. The problem with that is that the cloud connectivity will drop at that point. If there's a DHCP address it can fall back to, it'll do that until you swap the providers. If there is no DHCP to fall back to, it may fall back to the old "safe" configuration as described here: https://documentation.meraki.com/zGeneral_Administration/Cross-Platform_Content/Behavior_during_Conn...
Either way, if anything goes wrong or if my assumptions are incorrect, you may have to access the local status page, which can only be done locally.
In short, it's something I wouldn't do without having someone on site just in case.