A lot of children benefit from having some structure to their days. This doesn't mean you have to regiment every minute, but just have some kind of rhythm to the day. Also, it may be a reaction to the very full days a lot of kids have during the school year; sometimes children who are too accustomed to having their days fully planned out for them don't know what to do with themselves when given a lot of latitude.
I would suggest giving him loosely-structured days, where there are a few fixed timepoints, or one or two small tasks to be accomplished (or fun events to be involved in), interspersed with one or two or three hour unstructured times.
With my own, I find that it's not so much free play that does it, as excessive screen time. Reading all day or building forts in the yard (or living room) does not make them cranky, just satisfyingly tired.
I also find that there was a difference when we went from one child to more than one. As much as having multiple children introduces the option of quarreling, they do entertain each other, which -usually- has meant net better behavior.
