Math teacher here. Fluency and speed are not the same thing. Just like being fluent in Spanish is not the same as being able to speak it really fast. Yes, if you are not fluent you cannot speak fast. But you can be fluent and not be a fast speaker. And, I might add, there is much value in speaking slowly and precisely and precious little in being able to speak fast. This is all analogous to math in the classroom.
The only thing that you really need to know 'cold' are your times tables. You use them for everything: starting with obviously all multiplication and division, but also fractions, factoring, exponents, and so on. Sure you need to know how to add and subtract without using your fingers, but I don't think you really need to memorize things like adding positive and negative numbers. I still visualize a number line when I do that sort of thing, so technically I am calculating, not regurgitating. And I am pretty good at math!
Obviously being very slow is a problem. Being average is not. In fact the students who work at a more moderate pace are generally more careful and make fewer careless errors. As the math gets increasingly rigorous, precision is definitely rewarded over speed. And both become decreasingly important in respect to the abstract ideas. As for computation, more than speed, kids need to be able to mentally multitask: Multiply and add/subtract simultaneously for example when you are factoring a trinomial. Fluency is critical for these tasks. Speed is not.
I teach high school and as much as I *need* kids who know their times tables and complain when they don't, I really don't understand all of these time pressure tests. If a child can successfully perform multidigit multiplication, long division and adding and subtracting fractions with unlike denominators then she knows her times tables. If I were teaching elementary, I would push on to those and let mastery evolve from the constant reinforcement those types of problems provide. It's probably a function of our annual testing cycle that requires 3rd graders to be 'perfect' with their times tables before they move on to the next thing in 4th grade.
By the time you are in 7th grade advanced math you should be doing algebra. There is no place for speed races at that point, especially with signed numbers. Accuracy, yes. Speed no.
If it were my kid, I'd be in there wagging my chalky finger at the teacher *and* the principal!
