are pings going out to other sites ?
can you ping say 8.8.8.8 ?
did you do trace route from client to your external IP ?
Try different packet sizes too.
100 and 1400 i use as testing.
They would be good start.
Then try the same from another location.
You will find often routing issues ISP - ISP in the path between their ISP and yours.
The issue if if a third party ISP, good luck trying to get them to fix the issue. But you can put pressure on your ISP to log tickets against their peering ISP.