Welcome!
Timed math facts have been an issue for us too. Last year (gr 2) DD had to do "Rocket math" where they needed to do 40 simply addition problem in a minute, if they didn't get it the first day they did the same ones the next day and the next and the next until they finally managed to get all of them done correctly in a minute. Well, after about 2 1/2 weeks on the same set of 40 questions DD was convinced that she was horrible at math and could not do it! Eventually we figured out that like your son she thought they were stupid and didn't see the point of having to do them that fast. She could do the facts, but wasn't interested in going quickly because she would have to write sloppily! When we suggested to just do them and get it over with, she too refused saying it was a waste of time.
Her teacher used this issue as an excuse to not give her other more appropriate work in math because "if she can't do simple rocket math then how can she possibly already know how to multiply and divide?" It was a very frustrating time for us. We finally convinced the teacher that she knew the facts and that as scientists DH and I would not allow them to continue to convince our daughter that she hated math and was no good at it. We got them to agree to decreasing the amount she needed to get correct each day and eventually her confidence started to return.
Good luck!