It was just firewall rules. The custom routes in the Vnet weren't being automatically included in the "VirtualNetwork" service tag because the routes aren't applied to the vMX subnet (and shouldn't be). You can use the effective security rules tool to see the subnets aren't in VirtualNetwork, so you need to explicitly make rules for them.
... View more