Thanks so much for your responses.

I am definitely weighing the pros and cons (I've been considering homeschooling for a year and a half already). I have recently been more encouraged because another mom who lives quite close is considering it as well and one of her 4 children is in my son's current class. Physical Ed won't be a problem as my son plays tennis twice a week and is in baseball and soccer leagues. I have been doing things with my son for years, but it doesn't seem to be enough.