have you replaced the trigger spring? A lighter one will make a significant difference. It does help me if I really wrap my hand around the grips - taking out all the slack. Then concentrate on holding the gun (like you say) between middle finger and thumb. I've also found that concentrating on the front sight helps - that's all that needs to be in focus - you can't focus on three different things at three different distances at once, so don't even try.
Disclaimer: most of my pistol shooting is for pistol FT from a modified Creedmoor position.