We have a son with similar issues...
Given the level of your son's interests, I highly doubt that a teacher will be able differentiate sufficiently (does a first grade teacher even know factorials?) Then again, having looked at some elementary school textbooks (albeit European) - have you thought about how many grades ahead your son would have to work at? Second/third grade likely won't make a big difference (especially as I would expect your son to learn any gaps quite quickly). Would there be issues in him having to explain his strategy in words, which might be a concern with kids many years older than him? Could you homeschool him just in math?