My answer is that it depends. Can he elucidate what-- specifically-- he objects to about the meal around the table?
Can you allow him to "opt out" of those elements in a way that feels okay to you both?
Honestly, I do think that not allowing him access to the cultural elements which sideline the religious ones has the potential to be hurtful. It's very reminiscent of my own teens, at any rate. Adolescents are highly sensitized to rejection. Even if they may blow it off in the moment, or seem hostile when it comes from family-- it hurts.
I don't know if compromise is possible here-- but if it is, I would strongly urge you to do that. Is it possible to conduct the ritual portion of your meal as a "prelude" to eating together as a family? I understand that it may just not be so.
