You can be spiritual without being religious. We aren't super religious but we do belong to a local church. Last night, my kids volunteered at a homeless shelter our church sponsored. They got permission from our scout leader to volunteer there as part of a scouting volunteer thing - they wore their scout uniforms to the shelter.
That is kind of your "duty to god." 1/3 or more of scouts are Mormon, so that may be where a lot of that comes from.
Your church/temple may sponsor helping at a food bank, or they have a work day at the church/temple where you pull weeds. That is helping your community ad doing your "duty to god."
There are a lot of ways non-religious people can still be involved in scouting.
