An anecdotal observation: When our son was 6 weeks old, we took him to a pediatrician, who had been practicing for almost 40 years had treated thousands of kids over his career (he has since retired). We knew him well because he had also been the pediatrician for our daughter, who was 3 years old at the time. After the 6-weeks exam, the pediatrician told us jokingly "don't bother saving money for college with this one, he will get scholarships for everything". He said this because our son was unusually attentive relative to the many other patients he had.
Since then DS, has been tested and found to be PG. Obviously at six weeks there was no unusual nurturing for DS compared to DD, so the difference was really nature.
My eldest's pediatrician said basically the same thing about my son at the same age. At checkups starting at 2 or 4 weeks, he would get him to do things like follow an object or turn to a noise, and then exclaim, "99th percentile!" My daughter was probably more obviously gifted as a newborn, and her pediatrician commented on her too (we had moved). My second was born early and didn't meet milestones the way his brother did until he was around 5 months old. His verbal abilities are better; math, not so much.
All I know about my IQ is that the tester told me I was past the 99th percentile. I suspect I'm well past it, based primarily on how long I can concentrate (many hours), how quickly I pick up things, and how much I can teach myself (and the results of my son's test, I suppose. Before it, I had always thought we were of roughly the same intelligence).
When DH and I talk about the cognitive strengths our kids have, we can see traits that they get from one or both of us; there are traits from both that seem to surpass what either parent has. This apparent synergistic effect may be related to environment, though, as neither of us was encouraged cognitively the way our kids are.
I agree about the high heritability of intelligence and also understand that there is an environmental component. A good environment can presumable help a child reach his full cognitive potential, whereas a bad one can presumably do a lot of damage.
I suspect that intelligence is complex and multifactorial, but that IQ tests are a good place to start.
