just an idea also, it could be intensity

many gifted ppl are intense/passionate about what they are doing, thinking, feeling and other children (or adults) may not know how to react to the intensity other than to withdraw...

best wishes