If you read Mikymike's 'solution' carefully, it says you need to bring both lines up and down manually in order to make this 'work'.
Although Meraki clearly says to not use this in production as they call it a 'bug' and this function will be removed in future.
So - what happens if you reboot with only internet connected to WAN1 and once it's booted up, connect the second line to WAN2?
This would be a manual line up/down thingie but as you noticed, your box will run into issues when rebooting and both working internet lines connected.
This (automatic) reboot might happen as we're still not able to control firmware upgrades manually - although many times requested here and wished on the list.