I would argue that most adults do choose their associations in large part based on IQ, but they don't acknowledge it or even realize it because the overlap between IQ and what people find interesting is so great. How many low-IQ physics Ph.D.s do you honestly think there are? My son found most of the kids he knows around his age who might be considered "intellectual peers" by participating in a community theater group doing Shakespeare productions. The "shared interest" was also a filter that selected in favor of higher IQ.