I wouldn't worry about it too much at three. Maybe try playing some easy board games where you count spaces like Candy land.

My 2e dyslexic kiddo had/has similar problem where he will know/be able to remember something really complex but then completely space something like the months in the year. When he was three he did not like letters or numbers at all which is different that your son so perhaps you're just seeing a quirk.