My DD12 took Algebra in 6th grade and is now in 8th/9th Honors Geometry in 7th grade. Her first grading period she had something like a 99.3 average.
My DD17 took Algebra in 7th - the normal honors/accelerated path and is now in her 2nd year of Calculus as a senior.
Neither has expressed any regrets - other than trying to figure out how my younger will meet the requirement for 4 yrs of HS math when she exhausts the courses by 11th grade.
It's obviously an individual situation, but neither had issues, and I view Math as a bit different than many of the other subjects. It's so factual/concrete that maturity levels, age differences, etc., aren't that meaningful.
Personally, I skipped a grade in elementary school and accelerated math, and never had any concerns - but that was long ago.