Cathy, I don't know if this will make you feel better or worse, but I have a friend whose daughter is also severely allergic to milk and peanuts. Actually all dairy. People just don't pay attention. When she was at my house my DD5 who was 4 climbed up into the cabinet and took out some gum and gave her a piece. And we had been watching them closely, it happened in a second. Luckily the gum had nothing she was allergic to .
Her school drops the ball all the time, they just aren't mentally programmed to be on the alert about the allergies all the time.
I am sorry you are in this situation, I can't imagine how hard this is, especially when they are little.