For MATHCOUNTS, you register as a homeschool. Here's the
MATHCOUNTS Handbook and the required
homeschool participation form. Any parents reading this who are interested in signing up their 6-8 grade homeschooled child should note this year's deadline is December 13, 2013.
You can participate unofficially in the
Math League as a homeschooler.
Your state or local NCTM may offer competitions open to homeschoolers. Here's the
directory to find your state and local affiliates.
Finally, check your
state law. Your child may be able to participate on your local public school's math team.