DS6 is subject accelerated into a 4th grade class for math (he is technically in 2nd now, but skipped K). I think if everyone is on board it can be very successful. For us, both teachers and administrators are very much on board. Both teachers want him to be part of their class (in fact I just heard he will be in 2 class pictures in the yearbook because he had them taken twice as both teachers wanted him included). The kids in the 4th grade are very accepting of him from what I hear, and what I have observed. They really care about him and enjoy him being a part of their class.
I think that if anyone isn't sure about grade level they should definitely do end of grade testing to see what a good fit is. You want him to be challenged in the grade he is in, but not have the work be too overwhelming or way too easy. It sounds like you already have some good communication going. Keep it up!
Now the maturity part I can relate to. Your DS may be like mine in this. My DS actually is quite mature, but he does act out and act immature at times (usually when he is overwhelmed or not stimulated and challenged enough). Sometimes he acts completely goofy around kids because he likes to make them laugh and sometimes isn't sure how to have a conversation with them as they are on different levels. So I actually think having DS more challenged and with older kids has helped this issue. My DS still struggles with getting out of his seat and randomly walking around at times, but overall he is doing quite well and the schools still know that is where he needs to be. I haven't really had interactions with other parents so not really sure about that part. I am sure other kids tell their parents stuff but it really hasn't been an issue so far. DS also has to go to a different school for math b/c 4th grade is in another building. DS6 is on the 4th grade Math olympiads team so I have gotten a few strange looks when I drop him off. But that's not a big deal, people are just curious and haven't seen this before. I would be curious too if I didn't have a son like mine.
And I think if they know it is a good fit, then they should get it started the sooner the better. Obviously I would try to not start him in the middle of a unit and have some support in place when it starts to check and make sure everything is going okay. I think the school knows how your son is doing in school and if they are suggesting it and you think it would be good to, then go for it.