I agree with the other posters. If he is happy than I wouldn't pull him out but I would look to supplement outside of school. Foreign language, chess, music, etc ... all good examples. And the reason for the supplemental is to give him a challenge. We all need to learn the art of learning so when the time comes and the work becomes challenging you won't hit a wall.

I also agree not to assume the gifted program is that advanced. I would do a little digging and find out the curriculum. Do they spend a lot of time on review? Just because the program has gifted in the title doesn't necessarily mean it is all that advanced.