Even though our school district offers accelerated math, some 5th graders still need to go to middle school for math, and some 8th graders go to high school for math. The solution is pretty easy if middle school starts earlier in the morning than elementary. Our district just holds the math class during first period. The 5th grader takes the middle school bus in the morning, then the district transports him to the elementary school. Due to the difference in school start times, the 5th grader doesn't miss much of the elementary school day.
If the district doesn't see this situation much, then maybe they haven't thought about how to handle it - maybe you could suggest the above?