Reassessing College Assessment I: the US News Rankings

By Richard Vedder, Luke Myers and Jonathan Robe

It is widely acknowledged that one of the cost drivers making colleges less affordable is the "academic arms race" resulting from a quest for higher college rankings, especially from US News & World Report (hereafter, USNWR) We have often claimed that the USNWR rankings are enhanced by spending more money and restricting access (by turning more students down), both of which tend to raise the cost of going to college. Additionally, rankings are influenced by evaluations of senior college administrators who often have not been on some of the campuses they evaluate, at least in recent times.

But is that correct? We have been engaged in a lengthy study of that topic, using multivariate statistical techniques like regression analysis to look at the relationship between USNWR rankings and various attributes of many different schools.

By and large, there is a statistically significant positive relationship between spending per student and USNWR rankings. Spend more, improve your rankings. If you are charged as a university president by the trustees "raise us to the next level in the college rankings," your first task is to raise a ton of money and use it to buy good students, lower the ratio of students to faculty, etc., things that are components in the rankings. You might also raise some more money by raising tuition fees by a good deal.

We at CCAP have developed for Forbes.com an alternative ranking that emphasizes performance and consumer satisfaction measures to a dramatically greater extent than does USNWR. Our statistical results also show a spending-ranking relationship. While we do not consider that a "good" ranking necessarily must be completely neutral with respect to spending (where spending does not matter), the current USNWR emphasis on that variable does contribute materially to the academic arms race. And in an ideal world the success of an institution would be measured on how much in the way of outcomes are delivered by schools for every dollar of resources committed. To date, that has been an illusive goal, in large part because of the intransigence of universities in providing good value added measures of performance.

Needed, of course, are even more indicators of the success of universities. It would be helpful if they had measures of the value added to their students while in college --improvements in critical thinking skills, knowledge of important aspects of the development of our civilization, etc.

One thing that most aspiring college students are interested in is the post-graduate success of students --do they get good jobs? Are employers happy with them? I am hopeful that the Boeing Company develops and publishes some sort of ranking of colleges based on the job performance evaluations of its employees, and that the U.S. Chamber pick up on it and encourage other employers to do the same. That would be a huge step forward and reduce the impact that the USNWR rankings have on the academic arms race.