Is the fact that you think it isn't hard the only reason why you reckon it's being taught badly, Iucounu
No, of course not. Without trying to be combative, I think I've already given enough information to get the idea across.
If time arithmetic were taught more the way similar arithmetical concepts are taught, stressing the conversions between hours and minutes, for example by setting up borrowing/carrying problems with hours and minutes columns (this is also a decent way to show how to work some problems with conversions between greater and lesser units of measurement), it would probably be easy to understand in the same way for the large majority of children. It'd probably be easiest for most children to postpone learning the modular aspects of time until the basic conversions are understood.
Aside from the understanding of what's going on numerically, the analog clock face can be confusing if improperly taught too. I think this is mainly because the hours and minutes are overlaid on the same dial; I've seen poor teaching methods begin stressing things like skip-counting or multiplying by fives to calculate minutes too early. Introduce all of this before the numerical concepts are understood, and it's a recipe for confusion.*
(Most of the rest of small, relatively unimportant time skills could easily be taught after a foundation had been correctly laid-- hour fractions 'til and past, etc.-- and thus should be initially postponed. I haven't attempted to list all the time skills here, or my take on a correct scope and sequence.)
I'm not a teacher, but I do have some math tutoring experience, as well as having taught time to one of my own children. I also have seen a wide array of teaching materials, and have bought some of them. These are the bases of my opinion. I'm confident that I could teach time fairly quickly to any child who's not developmentally delayed, and who's been correctly taught enough prerequisite math.
* There are a number of truly marvelous ways to counteract such issues, which this margin is too narrow to contain.
