We use a lot of nonliteral language here too. DS10 is pretty good, but not a native speaker at it. When he's upset, he reverts noticeably to being more literal.

Comics, as Ultramarina said, are a standby; we find that Gary Larson's Far Side cartoons (and Dave Coverly's Speed Bump ones too) rely hugely on inference, and are highly useful in practicing that skill. Plus we laugh a lot.

To the OP's question: we are people who seek to bring up the floor more than to push the ceiling for our kids; so we don't shy away from curricula that draw on our kids' less developed skills. In fact, a curriculum in the subject of the child's strength that also exercises the weak spots seems just about perfect from my point of view.

DeeDee