I also use the lowered voice (since that seems to indicate that you're not bragging) and "Yes, he's pretty bright" response. And then I change the subject and go back to my normal tone of voice, usually saying something positive about the other child.

As a response, it's true, short, doesn't belittle my child's abilities, and usually keeps things from getting weird.

The lowered voice is key. Say the same thing in a normal voice and you'd get weird looks. But lower your voice just a little, and people seem to be fine with it.

Kriston