These age and grade questions come up no matter what. One of my kids, very asynchronous in his development, has a summer birthday, and benefited from a later start to Kindergarten. He eventually had a lot of subject acceleration, but we never regretted the additional Lego-playing, fort-building time he got.
Anyway, he often got questions about why he got "held back." He would just tell them that's what his parents decided, or that he wanted more play time. Either way, people are going to ask and make judgments, and it's often HOW you react more than what you tell them. Good luck