I don't believe you need to be particularly smart to be able to do algebra. The fact that so many students cannot reach this very low bar is a big problem, in my opinion. I think the problem starts in elementary school. My kids did virtually no math at all from Pre-K until 2nd grade - yet in the same time they were required to read, write and have a very basic grasp of spelling. Starting in second grade they did 40 minutes of math three times per week (no homework given). One of those days was spent doing "math based games", which included checkers, chess, knot games etc. The remainder of the academic time was spent on literacy based activities. Kids are just not doing enough math in the early years to set them up for success later.

I am also a University professor. We are having a discussion about dropping the calculus requirement for the same reason. Kids just cannot pass it.