I'm wondering if he's a visual spatial learner. He doesn't like to show his work when he does math, he can't seem to memorize his times tables, his spelling isn't very good and his teachers have commented that he is performing below his potential.
Not liking to show math steps is a typical V/S trait. Inability to memorize times tables or correct spelling are not. V/S gifted kids usually excel at these tasks, because they store the information in memory in visual form, and can then recall the entire thing at once, in excellent detail. This is where the term "photographic memory" comes from.
I think it's possible that these two challenges represent a possible memory deficit. It might be worth exploring with a full psych eval. Then again, you probably already have an idea about this one way or the other, from observation.
Another possibility is that he's just not applying himself. This can be from a combination of being under-challenged in school, and from a negative self-talk that he's the "dumb one" because his brother is tagged with the gifted label and he isn't.
