Tuesday, August 25, 2009

Comparing Identically Ranked US News Schools With Forbes/CCAP Ranking

by Matthew Denhart

When browsing the latest US News college rankings one thing immediately jumps out. It isn’t the schools that make the tops of each list—as those are the ones you see every year—but rather the large number of schools that tie for the same rank with one or more other schools. Of the 133 ranked ‘National Universities,’ 113 share a rank with at least one other school, leaving just 20 schools with a unique rank. Indeed, Harvard and Princeton share the top spot while no less than 8 schools all come in as America’s 88th best university. It is a similar story with the list of the best liberal arts schools where 110 of the 126 ranked schools tie with at least one other; leaving just 16 uniquely ranked institutions.

Two schools sharing the exact same ranking suggests that both are of exactly the same quality. This obviously cannot be the case. While this would seem to be a point of criticism for US News, my friend and former colleague Luke Myers (who just recently moved on from CCAP to work for Senator Sherrod Brown) reminds me that the imprecise nature of college rankings in the first place somewhat justifies it. Any ranking methodology is bound to fail in capturing all elements of an institution to provide perfectly precise rankings. A personal taste of students makes this an even more impossible task. For example, a school with a small enrollment may be desirable for one student and undesirable for another. To account for this we worked hard with Forbes to allow students to build their own ranking criteria weighting various factors according to one’s own preferences.

What is interesting however is to examine the deviation among same-ranked US News schools using the Forbes/CCAP ranking. What does the Forbes/CCAP ranking say about the similarities—or lack thereof—among identically ranked US News schools? Do the two ranking methodologies agree that such schools are of highly similar quality?

For both the national and liberal arts college lists there are some similarities, but mostly differences. The variations in the Forbes/CCAP rank of schools ranked identically by US News are somewhat large. On the national university list, the average deviation in the Forbes/CCAP rank among schools US News considers to be of equal quality is around 27 ranks. For liberal arts colleges, it’s slightly less than 25 ranks. This suggests that according to the Forbes/CCAP ranking methodology there are rather distinct differences in the quality of many schools that US News ranks identically. On the national universities list consider Southern Methodist University, the University of Delaware and Worcester Polytechnic Institute. While all share a rank of 68 by US News, they are ranked 31, 53 and 203 respectively on the Forbes/CCAP list of top national research universities (note this is a sub-set of the overall Forbes/CCAP rankings that list both national and liberal arts schools together).

Jonathan Leirer showed the relatively high correlation between the two rankings in a blog post last week. Despite the correlation, this shows that there are many important differences as well. Prospective students should be somewhat cautious when reading the US News rankings before assuming that identically ranked schools are of identical, or even highly similar, quality. The Forbes/CCAP rankings offer an alternative approach that offers a valuable perspective on what makes a quality institution. We hope that students will use both rankings as an aid in selecting the college that best fits them.

No comments: