We had the same experience with a very experienced pediatrician commenting on DD when she was a tiny baby. At the time, I was weirded out by it because she was a tiny baby.

I tested as gifted, but with a very uneven profile. DH has never been tested (his schools had no gifted services), but his GRE scores qualify him for any high IQ society out there, though I'd consider that a rough measure myself.