Your son loves math; my father was a mathematician and LOVED music. I'm sure you've heard of the connection between the two subjects; have you considered piano lessons?
If so, please consider the Frances Clark "Music Tree" series. There is more than enough in there to keep any kid challenged. Piano pedagogy has changed in the last twenty years; we now teach kids theory, improvisation, composition,how to count difficult rhythms, as well as graded pieces. Kids now spend a LOT more time on the intermediate levels so that when they get to Mozart, Beethoven etc. they are so prepared.
DD8 is now at the early intermediate level (I've been her teacher for the last three years). If you give her a simple piece, she can transpose it into different keys after learning it (Frances Clark teaches one how to read music by intervals, which is very mathematical!). DD finds school easy, but her piano lessons are always challenging, and she's slowly, but surely realizing that she won't breeze through everything.
Other instruments are great, too. Piano is best for budding composers, because of seeing music on more than one staff.
