Number of national merit semi-finalists that a school produces seems to be a good way to compare a good school with another good school. Since students can only take PSAT once (in Junior year) and most people normally don't prep for PSAT, it may be a btter indicator than average SAT score (people can prep SAT and take multiple times).
Just a note - it is becoming increasingly common for students to take the PSAT in 10th and even 9th grade. Yes, that would be as a practice for a practice exam! This is more common at more affluent public and private schools. Also, some students do now prepare for the PSAT in ways that go beyond taking practice tests in 9th and 10th grade.
To the original poster - I guess it depends on what you want to find out. If you want to find out if there are smart kids at the school the PSAT is probably a decent indicator. If you want to find out if the curriculum is good I don't think it tells you that much.
