If by "push" you mean pushing obstacles out of the way so that your son can operate at his optimal level of challenge/learning, then I'd say as much as possible without making yourself an obstacle.

If other parents question the decision, then I'd pose the opposite question: "Why don't you hold back your little Billy? Or ask that he go back to K?"

For general adults who question the social compatibility... then: "Have a conversation with my son about anything and then come ask me again."