This reminds me of the fact that at HG+ levels (perhaps lower gifted levels too?) - without the necessary stimulation/challenge/environment you will see the older child not excel in their "specialist field"/ There are stats that back up that moderately gifted, or even bright kids more often than not outperform the higher gifted people in the workplace, academically etc.
I would surmise that it's a similar idea - either the dislike of the term gifted, the frustration of never fitting in, learnt underachievement - all pointing to lack of opportunity, environment and encouragement. Even the most astounding self-motivation can die in the face of lacking options.
I think the converse can be just as true, where the issue is a surfeit of options. The one track to excellence above average person fought each step of the way to their mastery level, and it is an epic haul to bring another skill to a similar place. So, they keep pushing with the same drive and tada.
Speculation...
Maybe "true" prodigies happen when they have extreme working memory capabilites and encounter the right topic with the right amount of neuroplasticity. Their brain rapidly starts hard-wiring into that topic, and it becomes central to themselves as even internal reward mechanisms are wired in.
