I'll add my 2 cents, I guess.
My MG sister has been through some really rough times in her life. At this point (in her mid-30s) she has little drive, no intellectual pursuits, and, unless you really push her, it is virtually impossible to tell that she is gifted at all. I wouldn't be at all surprised if some or all of your siblings have had similar issues. I know (because my DH is 2E) how hard it can be to grow up struggling and not having your needs recognized, much less met.
I grew up the HG+ child of two ND (well, one ND and one possibly MG) parents. It is hard, even though my family is pretty intellectual. But I think the gifted thing is a very small consideration in the big picture of our relationships.
