You might want to explore that report of hating it before finding another solution, especially since he always seems like he's having fun to you. I have a DD who can find and dwell on the negative in any situation.
For instance, when she was 4 I'd take her to a place full of big inflatable bounce houses. We'd be climbing all over the place for a good three hours, having a blast... but at one point, some little boy might have bumped into her. We come home, DW asks her if she had fun, and DD answers with an emphatic "NO!" When asked why, she tells about the boy who bumped into her.
If this doesn't sound like him, disregard.
