Ah, JenSMP, BTDT! I completely understand. You're describing me two years ago, just about to the day!
I'd bet that you're right that a good part of your reaction is the insecurity. I can say that because I have the benefit of hindsight, and I know how I felt then compared to how I feel now.

You know, I got into the only argument I've ever been in about our choice to homeschool VERY shortly after we pulled DS8 (then 6) for emergency homeschooling. The "friend" wasn't nice to me, certainly, and she absolutely WAS judging me. (Attacking me is more like it...) But if she said the same thing to me today instead of then, I'd just let it roll off me. I no longer feel the need to defend my choice, I know in my gut (not just my head) that we're doing the right thing for our specific situation, and I don't care what she or anyone else thinks. Today, I'd ignore her. But back then, just a week or two into homeschooling, I was still too raw to be attacked like that.
That's a long way of saying that I know how you feel, JenSMP, and it's normal.

It's also a long way of saying that things will get better. Actually, I've been amazed at how supportive the vast majority of our public school friends and neighbors have been with our choice.
It gets better.

HTH!