I'm coming to appreciate the "couldn't do it" comments more than I used to. Homeschooling isn't always easy, it's not always fun, and it can take a lot of time and energy. I take the "couldn't do it" comment as a nod to the effort expended.
I really do wonder sometimes about families (specific ones we know IRL - not a veiled reference to anyone here) who homeschool when it's NOT working out at all, when they have practically insurmountable obstacles, have to give up more time, energy and money than they have available, move heaven and earth... and it's still not working out. Really, in those cases I'd rather hear them say they couldn't do it.
In terms of its relationship to GT-ed challenges, I've always started from the stance that no situation is perfect. In our case homeschooling is a heck of a lot better than our other options. But there are options in other places that would be a heck of a lot better than my homeschooling... and the options we have here might be better than homeschooling for some families, or for some kids in some families. In our case I expect we'd be moving heaven and earth to get the schools to be a good fit for DS... and it still wouldn't work out.