It has been suspected of being linked to Asperger's, but AIUI in fact that idea has been quite carefully investigated and found to be false.

For me it's definitely not that I don't look at people. In fact there's an interesting online test for prosopagnosia (also tied in to research) which will let you see how you do in a test situation so you can be sure you pay attention. Try the Online Cambridge Online Face Memory Test from here:
http://www.faceblind.org/facetests/index.php

Email: my username, followed by 2, at google's mail