Just be warned, machining is just as addictive as air guns

. I buy most of my machining material here:
http://www.onlinemetals.com/ Not the cheapest, but they have a good selection, and they ship fast. McMaster-Carr is also a good source, both for material and tooling. I use the black acetal, but I don't really think it matters much. I also use Nylatron, it's a little harder and slicker than the Delrin. As to size, just depends on what you're working on, I try to have a variety on hand from about 1/2" up to 1-1/2". I also keep a bunch of aluminum on hand, both solid rod and tube, and in pretty much the same variety of sizes. Only a little bit of brass, it's pretty expensive, and I don't have a lot of use for it.
If you don't have cutting tools already, don't waste your time on carbide, get high speed steel. Insert tooling is expensive but convenient, grinding your own is considerably cheaper and offers a lot more flexibility, but you need a grinder, then you need to figure out how to use it. There's a couple reasons there why I'm still using inserts

. Later.
Dave