I made the assessment that the overhead of managing specific firewall rules for Teams Room setups was really not worth the effort, so I just stuck them on an internet only network with very lax rules.
May not be the best idea but was enough for my scenarios.