I believe you don't need to remove it, if it is a PSK SSID, when changing the password they will no longer be able to connect.
If I misunderstood, please provide more details.
I am not a Cisco Meraki employee. My suggestions are based on documentation of Meraki best practices and day-to-day experience.
Please, if this post was useful, leave your kudos and mark it as solved.