Monday, August 30, 2010

On the Usefulness of Rankings

By: Christopher Matgouranis


As it is college rankings season, this morning USA Today prominently ran a piece today addressing rankings. Within the last month, almost all of the major college rankings (CCAP/Forbes, US News, Washington Monthly) have been released, along with loads of after the fact analysis and occasional griping.

In his op-ed for USA Today, Washington and Lee University President Kenneth P. Ruscio speaks out against college rankings in general, essentially stating that they have gone too far (ie. Ranking the strictest campuses & Easiest to Get Around) and that many universities place too much stock in their value. Mr. Ruscio immediately denies the social benefits of college rankings (informing and holding universities accountable), claiming that rankings organizations purport this belief in order to help themselves financially.

Ruscio later goes on to say that:

"Rankings are not evil. Students and families need information. Four years of undergraduate education is not a trivial commitment. But the rankings game is on the verge of parodying itself. Worse, it threatens to drive strategic decisions on campuses in ways that have little to do with what should be important.

The most worrisome feature is that the frenzy feeds the bumper-sticker, attention-deficit syndrome in our society, a trend that higher education should forcefully resist. Not everything that matters can be measured; the most important things in life are the least susceptible to quantification.


Bits of data do not define the best college, no matter how much they are manipulated into the appearance, but only the appearance, of order and symmetry. Complex judgments about quality should be, well, qualitative. And they should be personal, informed by data but backed by intuition and self-awareness."

In this, Ruscio expressly believes that consumers (students/families) need more information but flatly denies rankings ability deliver this. I ask of Ruscio; realistically, do students currently have any other adequate means of gaining information? If his idea about an alternate way to evaluate schools was undertaken, who would conduct a qualitative analysis of universities? Accreditors, some other outside organization, the university itself? While I fully acknowledge there are limitations to data (and any set of college rankings), right now objective data provides the best way to compare between universities. Universities themselves place enormous weight on objective data in the admissions process. There are limitations to the ACT/SAT but at least they allow an apple vs. apple comparison. Hard data such as graduation rates, default rates and debt loads, (measures which the CCAP/Forbes rankings use) perform a similar function.

A blatant lack of transparency typifies the average American university. Most do not release readily available outcomes reports on their graduates, or internal student evaluations of professors, or a host of other information that prospective students may find useful (see a recent opinion piece in Forbes on this topic). Mr. Ruscio and his peers are in a position to change this. In general, requests to increase transparency and release more data have been ignored or denied by establishment types.

In response to this information void, alternate measures such as college rankings, ratemyprofessor.com, payscale.com etc, have arisen. People want information. Ruscio agrees with this. But to take his line that numbers and data used by these ranking systems cannot provide a useful look at schools would be a terrible mistake. The vast and diverse sets of rankings flooding magazine and websites of late allow consumers to easily compare universities overall and in areas that are important to them. Our CCAP/Forbes Do-It-Yourself Ranking goes one step further, allowing individuals to select the factors that they themselves believe are important.

Rankings (and independent outside sources such as ratemyprofessor.com) will always provide an important function by informing the public. Until universities begin to lift the shroud of secrecy and collect/release more information, they may be the only and best thing for consumers.

2 comments:

RWW said...
This comment has been removed by the author.
RWW said...
This comment has been removed by the author.