I'm just not sure if at all and how much I should worry about it now when his former classmates are just being introduced to addition to 10 and subtraction from up to 6 and for the most part have no idea what any of the math he's doing is all about.

We do mainly unschooling for now so whatever he's learning is his decision. I'm there to help when he asks me to help but most of the time he says "I don't want your help! I want to figure it out on my own!" which is exactly how I was as a kid. I couldn't stand anyone trying to explain things to me. I couldn't remember anything they told me. I had to figure out my own ways.