We have a 5 yo who attends Math Circle with 8 and 9 year olds, goes to math in the Grade 5 class, and talks algebra etc with middle schoolers who share the building. At home he kept on sneaking to the basement and reading my university textbooks until I finally bought him some elementary books.
To echo aeh, take it one year at a time and follow your child's pace. My advice to my child's teachers is that they should neither actively press the accelerator nor obstruct him. Sitting in on Math Circle, it seems that the biggest challenge is with children just a couple of years older: he thinks their juvenile humor is hilarious and sophisticated, and is frustrated that he can't keep up with them when there are tasks demanding more advanced motor skills. Though they are not actively mean to him, they are also not always very conscious that he is younger and thus less robust both physically and psychologically. (imo this is just because 8 year olds tend to be a bit oblivious) The much older children are very kind to him and recognize and respect his abilities while understanding that he's still just a baby. A lot of older children have younger siblings, and innocence can be protective.
I do think it is helpful for a young math-oriented child to spend time around others who love math and are working at their level. This helps to foster both a collegial and collaborative atmosphere, and is much less lonely than reading a book or working on a computer.
Even if your child does keep up a blistering pace in math I wouldn't worry too much about deciding their high school courses or workload. Speaking as a math person, much of math through typical second year university courses really is not difficult for those mathematically inclined. Doing multivariable calculus, linear algebra and differential equations will not add substantially to your child's workload: it will be different math than the courses typically covered in high school, but certainly no more work. Things do get more intellectually challenging once you start getting into analysis and advanced abstract math topics, but even then it is about deeper thinking rather than more time or larger amounts of work. This is very different from content-dense arts or science subjects, where university level courses do require increasing amounts of time for reading and lab work. Math is about working smarter, not more. It's a search for elegance - the ideal proof is both succinct and complete.
