We brought in 1st 2nd grade math books for DS when he was 4. The teacher let him do those. If your DS can't read that wouldn't be a problem.

At home play his favorite board game with dice. Many dice. When DS 5.10 was just turned 3 we started this. Beginning with 2 dice and by the time he was 4 he would add up 10 dice with ease using a combination of addition and multiplication.