Hello @Tishman
Based on what you said, I suspect your Trafic Selectors (TS ; a.k.a. encryption domain) don't match exactly.
I.e.: MX side may have a /24 subnet while the other side has a /25 ; so it works if the other side initiates because MX TS is like a summary route. On the other hand, MX initiating doesn't work because the other side has a /25 more specific TS.
You may want to double-check with the network admin at the other side and make sure both sides are configured with the exact subnets as traffic selector (TS).
If you found this post helpful, please give it kudos. If my answer solved your problem, click "accept as solution" so that others can benefit from it.