Make sure you distinguish between arithmetic and mathematics.
DD11 detests arithmetic 'facts' and does not memorize. This is different from remembering random trivia she saw in a book once, which she does all the time. If she had never seen math outside of school she would think she was horrible at it and hated it. Instead we've supplemented all her life with conversations and books about mathematical ideas - try The Number Devil, it's interesting for adults and kindergarteners alike because the ideas are not the math you encounter in school - and she loves playing with math ideas and how they fit together, just like her science interests. (We haven't yet found a way to keep up with her scientifically.) At the moment we're just ignoring the arithmetic facts (lack of) speed and moving on to algebra as quickly as the school will let us.
DH and I are both terrible at memorized facts. I did memorize in 5th grade but don't trust my memory; he calculates quickly to this day. I'm a scientist and he's an engineer.
At Caltech, there is a rule that the youngest non-math major calculates the tip because the more mathematics you study, the worse your arithmetic gets. I think it's true. The one time I violated the rule we ended up with about a 45% tip.
