I don't think that you can get him to love math and pushing will probably make it worse. At the same time, he is advanced enough in math that he should not have any difficulties with skipping one year and compacting the next two years. There can't be that many holes as there isn't that much to 6th grade math and I think that compacting pre-algebra and algebra shouldn't be too onerous, particularly in a classroom setting.
I wouldn't worry too much about the speed as long as he is accurate. I never pushed math facts and somehow both DS and DD picked up enough from regular math usage in school although DD is slow and barely passes the speed tests. Incidentally, DD also does not like math and received the same Explore score in math (75% for 8th graders) last year as a 4th grader. In her case, she is ready for pre-algebra but she is against acceleration so will study 6th grade math as a 5th grader in the fall. My point is that your DS can't have that many holes either and should not have any problems with compacted 7th/8th grade math so better not to exacerbate his resistance to math.
