Even with the math schedules align, there are bound to be classes missed now and again due to field trips, assemblies, state/county testing, etc. I think it should be fine if he misses some of the math classes because the accepted practice is to only accelerate to a level that is at least still a bit low relative to ability. Math was still really easy for DS each time after his SSA so he can focus on executive functions and writing demands (Pre-Calculus in 8th this fall and missing classes has not been a problem). I would focus on him not missing any of the fun stuff with his grade mates.
