Thanks for all these responses.
He *is* on the cusp of reading. He's shown very little interest in books from the start, but I've recently found that he likes scary books, so we've done that. It's almost to where he'll have a story memorized and be able to "read" it that way. I've also been seeking out books for his baby sister that he can "read" her, ones like nursery rhymes where he already knows the words.
In all honesty he hasn't shown a whole lot of interest in learning to read Japanese, but DH feels a pressure/guilt/?? to improve his Japanese. The Japanese style of education is very top-down, "You must memorize this." and of course, that's the way that DH teaches him. I can see DS getting upset when he gets an answer wrong because DH is not pleased with him. I guess there's a part of me that thinks the more I can teach DS, the more we can avoid DH teaching him, because I don't think that style's so good for DS. DH seems to have a very difficult time reading people's non-verbal communication and knowing when to stop pushing.