Wow, he sounds like an interesting kid to live with!
By definition, ADD/ADHD has to manifest itself in more than one area of life: school, home, sports, etc. Not necessarily ALL areas, but more than one. Also, being able to focus on one particular item of extreme interest doesn't rule out attention disorders.
It surely does sound like he's a bright kid, and the fact that he has no problem with attention in his math class is reassuring. He might do better with some more advanced work in other areas -- and you can use his IQ scores, when you have them, to help you advocate for that.
On the other hand, you mention that the math class is very structured. It can't possibly be the first "very structured" environment he's met at school. Does he do better in other more structured settings, or do you think he's doing better because the math is challenging and interesting?
The disorganization and leaving schoolwork/books in the wrong place could be ADHD symptoms, or they could be "9 year old boy" symptoms. :-)
I'd suggest some harder work to start, and see if the inattentiveness improves before moving on to the next step. You can always get a copy of the Connor's Scale from your pediatrician, if s/he is experienced with attention disorders. And if there's any question about a diagnosis, a psych eval can be helpful, albeit expensive.
Good luck!