I Maybe because I want him to want more, to push himself harder, maybe not to be like me, have bigger dreams. ... Why do I feel that I am failing him, that I am not helping him to reach his full potential?
You feel that way because you are accurately observing the situation. Chess isn't the way to go, for the reasons you stated. Your child needs to be in a classroom that is challenging, or, in the beginning, if not actually challenging, then at least engaging, and then slowly turn up the heat - at school- to get to challenging. (I'm assuming you child is at least 5 - 3s and 4s are allowed to be fickle.)
As you pointed out, if you keep you child with agemates, he may not be challenged until graduate school. You have the advantage while he is young of many more possibilities to get challenge in there.
Advocacy it a hard road - I suspect you will find the challenge you need to grow into your gifts along it as a side benefit.
Love and More Love,
Grinity