I can't say that I have every had a problem with flies taking over a feeder. I do have one land on a feeder every now and again but they don't stick around nor have they ever deterred hummers from using the feeder.
What type of feeder are you using?
USDA hardiness zone 6
Heat zone 4
Sunset zone 39