I don't think any reason not introduce him math fact

I did it when my son was 2, soon he mastered all addtion and substraction, then I have to introduce him the times table by the time he was 3 and half, he absolutly loved it, then division, fraction.....

Follow his lead, if he like it, provide the books or whatever, he will tell you when he is ready to learn