Our DS4 is more verbal than mathy at this point, but I can remember precisely how old he was when he started counting to 20, and why. We were visiting relatives out of state and grandma bought a 50 cent toy for DS at a thrift shop - LeapFrog Count and Learn Math Desk, which got loads of use by DS for about 6 months. There was a "let's count" game -- let's count to 20. Count with me..." One of the games on the easy level included "find the number 14," "find the number that comes after ___," or "find the number that comes before ___" etc. Now that I think of it, there were also adding and subracting games that he never played - i should pull that out and see if he's interested now.
We did have some flashcards for the alphabet, because I knew he would like them (and he did). I say if you think your kid would like them, it's not hothousing. We probably won't do math flashcards, unless DS4 all of a sudden has more interest in math. (He did ask me out of the blue the other day if zero is an even number, so maybe he's wanting more than the nothing we're doing now!)