Yikes. I hate it when a teacher unknowingly or negligently sets up a kid to fail.

But since you have documented the teacher's non-compliance with the 504, and discriminating against a kid with disabilities, even one with high intelligence and achievement, is illegal, I think you've got the better ground here.

If they don't give him a chance to hand in or redo those assignments, they are just plain wrong. If they try to remove him from the program because of that, they are just plain wrong.

And of course, I totally sympathize too. You don't want to be the "annoying parent" who complains to the school all of the time. But you have a track record of mistakes, non-compliance, and apparently a lack of problem-solving effort.