QOS for icmp won’t help everything else. I need to get the packet delays through the vpn down to the 40-50 ms range.
It looks like either the link capacity is not always 3-4Mbps, or more traffic passes through the vpn tunnel than you think it does.
As suggested for troubleshooting reason try to apply QoS on icmp and keep pinging. If the problem is fixed, apply QoS on any flow you wish to prioritize.