I went the invite everyone route and the pro is DD always got invited to parties (a big deal to me since at the time she had 2 moms and we lived in a conservative town). Plus, I like to be inclusive and hate for anyone to feel left out.
But, if there's a kid whom your DD doesn't want there, I'd take in cookies or cupcakes to the school, then invite whomever will make your DD the happiest. At 3, your DD is unlikely to talk about the party too much and even if she does, the kids won't remember it the next day anyway. You might want to ask about how much extra kids are. Sometimes it's not much.
Feel free to disregard this advice given it's coming from someone who is about to send out evites for my DD's 7th b-day party. Between the neighborhood kids, including the nextdoor neighbor who is 2 years younger than everyone else and I feel I have to invite, and the only 2 girls in DD's girl-skewed class she didn't want to invite are the 2 she sits with, I'm saying the heck with it and inviting everyone...just like I did in preschool.