Yes, that is so true. Here in California, we rank dead last out of the 50 states for funding IDEA.
All we wanted/needed for my son's hearing loss was the FM system, which is relatively cheap for them to pay for, and the school did. His IEP doesn't address his giftedness, but we have an excellent gifted program that starts in 4th grade that he will hopefully get into.
I mention the above things since I didn't understand how it works. There is one educational pot of money. Funding for special ed comes out of it, leaving less for the general ed fund.