I know there are issues of legality in some cities / states, but I kinda went through the same issue with my Jeep. I have a '82 CJ7 that I leave the top and doors off of for the majority of spring and summer (gets hot in Texas!!), and for a while I had a pretty nice stereo and speakers in it. I even had the speakers bolted to the top of the wheel wells.
The head unit had a removable face, and I always removed it. But all it took was for someone to actually tear the speakers out of their boxes for me to scrap the whole setup. (Well, that and I was always worried that our brief thunderstorms that dump 2" of rain in 30 minutes would ruin everything).
So I invested in an Ipod. The sound was better, I wasn't competing with outside noise. And it certainly was portable.