We went through this with DD (just turned 4) and ended up pulling her out. She was so miserable. We are not sending her to pre-k for fear of it being the same thing.
As a child, I only went to a mother's day out program once a week as did most of the kids in my area. We all survived just fine going into a 5 day a week kindy program. I'm sure there will be an adjustment period, but I think you would have that regardless of how many days of preschool he is attending or if he even attends at all.
Now that DD is at home, we have realized that homeschool may be a very real option for us in the elementary years.
