I would not say that it's a bug, just that it is an unexpected "feature" of the Meraki implementation. At least it is a documented feature.
And MTR is again the wrong test as it also uses ICMP and the MX will ignore the flow preference for it. I think that all these tools that show the public IP are the right thing to test it. Or to do a packet-capture on the secondary WAN and see if the UDP/TCP traffic leaves this interface.
If you found this post helpful, please give it Kudos. If my answer solves your problem, please click Accept as Solution so others can benefit from it.