You have the option to remove and keep the settings, but it is highly recommended to perform this operation within a maintenance window.
I am not a Cisco Meraki employee. My suggestions are based on documentation of Meraki best practices and day-to-day experience.
Please, if this post was useful, leave your kudos and mark it as solved.