Thank you everyone for the input.
I resolved the issue, my setup was right i just had to allow Port 8100 in the VM firewall.
Worked it out by using wireshark
For those wondering what i was setting up it was a cloud solution for an onprem application that pushes a print to a onprem server. Everytime the main site went down the other sites couldn't access the Server on the VPN.
I recreated it with a Azure to Meraki VPN that then pushed an application print to an Azure VM that had an IIS server connecting to a web server, running as a Print Proxy Server. it then pushes the print to Universal print and back to a site.
This also allowed the application to be installed on PCs from Edge on the VM public IP.
Now each site runs individually and access the same functions in the cloud.