after you are done grinding the flavored coffee beans, there will be residue left behind on the burrs. If you don't want subsequent coffee grounds to taste like "pumpkin spice" or whatever, you'll have to clean everything out.
One method I have read about is to run uncooked rice through the grinder. Apparently, the grains of rice are able to push the remnants of the coffee beans out the grinder's chute.
http://www.cigarsmokingman.com