That sounds like a great plan.
I would encourage you to find local homeschoolers in your area that are familiar with state laws. If you can find something to do with your son during the day - you can focus on the math just on the weekends if that works best for your family.
This is very exciting!!!