Even if it is not your intent he may be feeling shamed and confused. A lot of what two year olds do "wrong" comes from either not understanding or not having the impulse control to avoid an offense. He gets he upset you but that doesn't mean he's better able to avoid it next time.
You may find it works better to positive phrase what you want to happen and make sure he understand it. Instead of "stop throwing the ball at the baby and say you are sorry. "The ball goes outside, do you want to put it outside or should I." Instead of "don't hit the dog" "the dog wants gentle pats, show me your gentle pats."
I see a lot more downsides to forced apologies that benefits. I think part of why so many adults really struggle with apologies is they associated them with shame they felt as a young child forced to apologize when they didn't even understand what went wrong.
And, I agree with the suggestion that a lot of it is about prevention. If you post some specific stuff you are having trouble with I bet people will be able to come up with more concrete suggestions.
