The hardest thing for me was 12 years of relatively decent gifted education led me to believe the world was full of people just like me, and that I'd be able to do anything.
I never learned to focus on what I truly enjoyed and was good at, nor that for a lot of people in the "real world" another person's intelligence and giftedness is a threat, not an asset. As a result, I feel like my career has been spent drifting from one "oh that sounds interesting" to another without ever having found something that fulfills my mind and soul.
