Rankings Revisted

By Richard Vedder

The Times Higher Education (THE)world university rankings are out for 2009, and the big news to many is the decline in American listings in the top 100 schools. To me, the news is how much difference there is between alternative rankings systems, and the implications of that.

The correlation between the American universities included in the THE rankings and the FORBES rankings is a very low .491. Cornell University is ranked by FORBES below Cornell College, not in the top 100 American schools, while THE ranks Cornell the 15th best school in the Solar System (or at least Earth). The correlation between THE and US NEWS is higher, but there are still some big differences with respect to American schools in these two rankings.

For the record, I think the THE rankings are based on a bad methodology. The best thing about them, as Daniel Bennett has observed, is the employer survey, a good idea.. But reputation counts for a huge portion of the THE rankings, the sample of academics surveyed is biased, there is a pronounced pro-British bias in the rankings, etc. If an American school hires a lot of Indian mathematicians who speak lousy English, it will raise that school's THE rankings, but almost certainly lower its FORBES rankings. Foreign is beautiful to THE, not to Forbes (which is officially neutral on the geographic origins of students and faculty, as is US NEWS).

Forbes main interest is consumers and undergraduates, THE's is research and what academics think. US News is somewhere in between. For the average citizen, a "do it yourself" approach to rankings works best --possible on the Forbes.com web site. Emphasize the criteria that YOU think is important, not what some periodical thinks. An expansion of the Forbes do it yourself data base would be worthwhile.

All of this points out that perceptions of what constitutes "excellence" vary immensely. It also points out that we have very poor and imperfect measures of success or output in higher education. If we could measure value added by an institution with respect both to teaching and research, we would have better rankings, independent of the value different individuals place on different things. To THE, research seems to play a dominant role, to the extent that not a single liberal arts college in the U.S. is listed. Amherst and Williams and West Point --all ranked highly by both Forbes and US News, are non-entities to the gurus from London (who manage to get two London based institutions in their top five).