Just saw your note about being caught off-guard. One thing that you can do is to choose a canned response that you can always use when you're caught off guard, to postpone the conversation until later when you've had time to think it through and prepare. For instance, you can always say "Ms. x, I appreciate your question and want to be able to discuss this with you further, but I"m in the middle of something (or on the way to somewhere) - can we meet to discuss this later? etc.
Best wishes,
polarbear
