Goh,
Try this product called PEC-12. I have never used it on a baseball, but this stuff is great for taking off residue from photos. It will also remove silver sharpie or the color sharpie ink right off a glossy photo with no damage or residue(just dont rub too hard)
Below are the links to the PEC-12 and the PEC PAD.
http://www.lightimpressionsdirect.com/servlet/OnlineShopping?DSP=50000&PCR=30000:150000:155000&IID=5611
http://www.lightimpressionsdirect.com/servlet/OnlineShopping?DSP=50000&PCR=30000:150000:155000&IID=7239
Hope this helps, Bobby