I highly doubt it! I just read an article about a local kid who started piano at age 12 and is now being requested to perform internationally before age 30. He was quoted as saying a lot of the competitions he was in were against kids that started at 3, 4, or 5. He was just very focused and motivated when he started.
My kids started music early - but as a way to work on small motor, focus, and perfectionism in my son. And for working with a mentor, accepting criticism, and perfectionism in my daughter. Learning music has been secondary to those goals!

My 9 yo has become fairly accomplished for his age, but I had no expectations. There are a lot of ways to learn these lessons.
I don't know how old your child is, but I wouldn't worry about it. Pursue music if and when it works for you!