Hi Kriston -
I wanted to say that if you start a homeschooling class in "memorization" don't put 'math facts' as the first thing on the agenda. Let him know that he won't always get to choose, but build up those drill muscles a bit before you go back to an area that he already feel bad about.
Of course 'important phone number' is a family favorite, and does have numbers, you can brainstorm with him what he would like to learn, and let him choose the first 'clump' or two, then important phone numbers, then a chunk of the math facts, then something else - perhaps the location of type writer keys ((red face)) or "Miss Mary Mac?"
Also - see if you can get him moving or using his hands as part of his memorization. Maybe a kind of arobics were each number is a movement, gross motor or finger play, and then the phone numbers (and later the math facts) become a kind of dance?
Best Wishes,
Grinity