What about anxiety? I feel like you might be describing performance anxiety. One understands the concepts and can complete problems pretty well at home but has trouble when put on the spot. Feeling stressed impairs the ability to focus on small details, and impairs the ability to access what ought to be automatic, can make it feel hard to even add single digit numbers. It's a vicious cycle, the more anxious the worse you do, the worse you do the more anxious you are about the next time. Performance anxiety would also fit with ADD meds making her feel unpleasant, they tend to make anxiety worse rather than better. She may have no idea if she does have performance anxiety, at her age she might think that was how everyone feels during tests or when put on the spot. How has she felt during other performances, school plays or music performances in front of a group, does she enjoy that kind of thing or get butterflies? Although having asked that, performance anxiety can be limited to just one area, it's going to be worst wherever one feels one is doing the worst and can be non-existent in an area one feels proficient in, for example a sport one is good at.
One thing that has been used for performance anxiety is beta blockers, very popular amongst public speakers and concert musicians. Beta blockers such as propranolol temporarily block the action of adrenalin on the heart and (some) on the brain. The same doctor that prescribed the ADD meds might be consulted, or a psychiatrist. I can't find a great discussion of this on line, mostly when one googles beta blockers one gets the use in high blood pressure, but here's one link.
http://articles.latimes.com/1987-06-07/local/me-920_1_test-anxietyLong term some specifically targeted stress training with a counselor/psychologist such as breathing techniques to avoid hyperventilating, positive self talk, learning how to take mental breaks, etc might be helpful if there was an anxiety component.
Whatever is causing it, it does sound like more is going on in some way than just the lack of automaticity, more than some oops mistakes noticing the plus or minus sign. Those things alone wouldn't usually lead to Ds and Fs. Personally I have a gap where my analytical ability is higher than my working memory and processing speed and I make stupid mistakes, and can't keep track of too many things at once. But a calculator and some scratch paper and a little good advice on slowing down and keeping track of problems I had skipped over, etc, and I sailed along fine... So there just feels like there has to be more than that alone going on with your DD.
What's wonderful in all this is that she hasn't just given up! In her spot I'd surely be crying under the covers and refusing to go to school at all. She clearly has a lot of internal strength to be still trying at these subjects, despite the awful test results, the family stress, and having an older sister who has a much easier time. Any one of those issues would seem like a good enough excuse to many kids to quit trying.
