We have the same sort of situation because of homeschooling. Tapdancing around it for too long makes it seem like we're ashamed or something. And as with you, it's going to come out eventually. Why hide it?
I have had my best luck with being direct and honest, but the less said the better. If I say two sentences in a row about our situation without someone asking specific questions, I know I'm talking too much.
Most people are just being polite. They don't really care. I try to keep that in mind. Short answers given with a friendly, confident smile, followed by a quick change of topic seem to go over best for me.
The confidence is key: people generally only pursue if they sense that I am uncertain about the choices we've made. And practice in front of a mirror. It helps you know when to quit talking!
