"Cautiousness" is another proxy that others seem to use to identify introversion in others, particularly in children.
That might explain why some children seem to be introverted right from the start, and most of them stay that way, whereas some kids SEEM to be extraverts who later convert to a more cautious and standoffish approach. I think that the latter is probably especially common in gifted persons, as we learn that most people don't get our jokes or find the same things interesting-- they just think we're weird. So we tend to spend an extended time in "observation" mode, which comes across as socially anxious or timid.