I think the middle class started to retreat through the recession of the 70s, when the Japanese started eating up our automobile industry.

Muliplier affect of the autos was 1:8. So if you look in your driveway and see a non US car, you contributed to the decline of the middle class.