Anyways, some of that is a long way off, I'm getting too far ahead. Hang in there! Look for interest based activities like library story or craft times (and check libraries further away, the closest may not be the best). Just because it says for 4 and 5 year olds doesn't mean you can't just show up, there are usually a younger sibling or two in attendance anyhow.
I agreed with much of what you said prior to the recommendation to visit library story time. That was a disaster for me and DS when he was three. He refused to sit and listen and instead wandered the room wanting to touch the fire extinguisher, the CD player, etc. In retrospect, I think the reason for this was that the storyteller wouldn't let DS see the book up close enough so he could follow the words as she read. She kept telling him to sit back down. (DS had started to read easy readers on his own at that time, so of course he wanted to follow along!)
When it came to "share time", he insisted on bringing his compass. He explained exactly what it did and pointed out the needle was facing North. (The correct direction at that time.)
The other mothers bit their tongues and avoided any interaction with us. I went several times hoping something would change, but sadly had to leave partway through more than a few stories because DS was "disruptive".
Having a gifted child is isolating, period. School didn't help us much. DS cried that the kids didn't understand him and his teacher reprimanded him for his perfectionist-driven crying over his performance. She also reprimanded US-implying we were pushing him at home!
I tried to connect with other parents of gifted children in my community through our local online chat, but found no one interested. Maybe you'd have better luck.
