I've never had my parents friends show up for one of my kiddie birthday parties. Unless their child is coming or they are helping with the party it's not necessary for them to attend.
I would only invite friends if they are close with your child. Otherwise the child might feel like the party is not specially for them. Also I know my child gets uncomfortable when there are a lot of adults around. Keep it mostly children. So for the age thing, I would stop inviting your friends once your child is old enough to have some of their own friends.