I would take the advice with a grain of salt. What is the most advanced math course at your HS? Calc BC, Linear Algebra, Multivariable Calc? Your kid is not there yet. To do "real" physics, you need calc. And most college courses labeled "College Algebra" are for the kids that didn't get math in HS. You don't want him in that class.

Continue to encourage him and continue to give him opportunities - but don't send him to a college class for a few years.