My DS10 started algebra informally with me around his 8th birthday, and then went into a serious course in school at age 9.
I wouldn't say it's been too difficult for him at all. For example, his total algebra exposure over the summer break was ten days of review right before school started (around an hour-ish per day). He tested into the next level with a high score on the pre-test. So he obviously didn't forget it, which I would have expected if he'd been too young for it.
So, FWIW, that's one anecdote in favor.
Val
