My five year old (in K, with subject acceleration) attends a weekly math circle for children in Grade 2 and above. He's the youngest in the room, which is a factor from a maturity perspective and for anything involving fine motor skills (e.g. folding, precise drawing). But he loves the content and the off-curricular math. They have covered topics including hexaflexagons, transformations of plane shapes (e.g. stretching in one or both directions), Fibonacci series mod 2,3,4,... etc. For our math circle fluency in basic arithmetic has been sufficient for most exercises.
The 2nd - 3rd grade boys in our math circle are typically fairly wild and disruptive, so you might want to consider social factors. (My five year old wants to copy them) My observation has been that the girls and older boys set a better example of appropriate behavior. But even with the bigger boys' roughhousing I think it's good for my son to be around other children who enjoy math and who are quick to figure things out.
