Honestly, I would keep him in it. Let him develop some extra emotional resilience in dealing with a negative person and work with him on ways he can learn to deal with his swim teachers attitude.
However, I think that periodic meetings with the instructor to fill her in on how you know your son learns best and strategies that might help her along would be useful.