I would consider things like the size of the kindergarten, how welcoming the school is and how likely it is that your son will have to be skipped at some point anyway (it might be less disruptive to front-load the skip).
My DS started at 4.5 and did wonderfully. However, his kindergarten had only 9 kids and the gifted-friendly school was fully behind the early enrollment, so DS got a a lot of individualized attention and when he did occasionally have an age-appropriate meltdown, it was no big deal.
DS did eventually need to skip another grade later on, and having the early start made a 2-year acceleration seem like only a 1-year bump up, which made the later transition easier socially.
My mom refused a skip for me because she wanted to keep me [unchallenged] at the head of my class, but DS has shown the folly of this thinking. Even with the early start and another skip later on, he's still at the head of his middle grade class in many ways, both socially and academically. His teacher recently called him the most mature student she has!
Can your son spend a day in the kindergarten and see how it goes? My son visited for a day at age 3.5 and it went so well that all involved agreed early enrollment was a no brainer.
