I agree with VWmommy.
but it is not just about not developing an ego. but developing a healthy ego instead of a sense of entitlement or elitism. a healthy ego with appropriate self esteem can also help avoid the perfectionism trap.
if everyone expects him to be smart all the time and he learns to get his sense of self worth from that, it can become difficult to acknowledge and learn from mistakes. In really smart kids this can begin really early. so no 4 is not too soon to teach a healthy view of the world and ones place in it. Even younger is is fine too if the kid is self aware early.