Wednesday, August 13, 2008

Rankings Commentary

by Richard Vedder

With the possible exception of buying a home, choosing a college is the biggest non-financial investment a typical family makes. Because colleges usually do not provide good information on their success in educating students, and because even the cost of college is a bit hard to calculate given the vast array of scholarships, loan options, tax credits and the like available, organizations help consumers by ranking colleges and universities. Forbes, working with the Center for College Affordability and Productivity (CCAP) now offers its rankings of 569 institutions to give students and their parents additional assistance.

Other rankings rely partly on the judgments of high level university officials or even students. They also consider whether the school provides lots of instructional resources, or how selective it is. We put ourselves in students’ shoes. In picking a school, we believe students ask several questions: how good is the instruction –will I like and learn from my professors? Will I have a good chance at becoming successful after graduation in my chosen career? If I have to borrow to pay for college, how deeply will I likely go in debt? What are the chances I will graduate in four years? Does the school I choose have many students and faculty who demonstrate academic excellence that is recognized nationally or globally?

The CCAP staff (mostly college students themselves) gathered information that helps answer these questions. We based 25 percent of our ranking on averages of literally millions of student evaluations of courses and instructors from the ratemyprofessors.com website, and another 25 percent of it on enrollment-adjusted entries in Who’s Who in America. The other half of the ranking was based equally on three factors: the average amount of student debt at graduation held by those who borrowed; the percent of students graduating within four years; and the enrollment-adjusted number of students or faculty winning nationally competitive academic awards like Rhodes Scholarships or Nobel Prizes (see the link for a detailed discussion of methodology).

Looking at our rankings, we find:

• Students strongly prefer smaller schools to larger ones; the median undergraduate enrollment in the top 50 ranked schools was 2,285, and only one school (the University of Virginia) had more than 10,000 undergraduate students;

• Strongly related to that, only six of the top 50 schools were public institutions, and three of them were the major military academies, and one (New College in Florida) a really unique public liberal arts college;

• Even looking only at national research universities, only four of the top 25 ranked are public schools. Among public research universities, Virginia and California do especially well, with the top two schools being in Virginia (the University of Virginia and William and Mary), while six of the University of California campuses are in the top 25 (Berkeley, UCLA, Santa Barbara, Riverside, Santa Cruz and San Diego);

• Several relatively unknown schools (e.g., Wabash and Centre colleges) do exceedingly well in our rankings, while some well known schools do much poorer than in the popular US News & World Report rankings, with three Ivy League schools not making the top 50 (Cornell, Dartmouth and Pennsylvania), along with such other prestigious schools as Duke and Washington University in St. Louis;

• Among the top 130 national research universities, the geographic center of our ranked schools is near Rolla, Missouri, only 72 miles from the national center of population; for the US News rankings, that center is in southern Indiana, some 300 miles more north and east;

• Nonetheless, nine of our top 10 schools are within 150 miles of the Atlantic Ocean; at the very highest ranks, the East Coast schools are still disproportionately represented;

• Schools with a distinctive orientation do well; four of the top dozen schools are either relatively small schools with an engineering (Cal Tech) and/or military (U.S. Military Academy) emphasis, or are single gender (Wellesley is female, Wabash is male).

• Both Boston (Harvard and MIT) and Chicago (Northwestern and the University of Chicago) have two of the top 20 institutions. Over half of the top 25 institutions are in three states: Massachusetts (six), New York (four), and California (three).

Why did some schools do very well compared with other rankings? Wabash College did very well on both the student course satisfaction and post-graduate employment distinction factors; Centre College was in the top 10 percent of schools on all criteria. Cooper Union in New York (ranked 42) far outdistanced schools like Penn and Duke by having low student debt along with many winning nationally competitive awards. Students at West Point loved their classes and, of course, incurred no debt. Students at Duke, Penn, Cornell and Dartmouth ran up large debts; additionally at most of those schools (notably Dartmouth), they were not particularly happy with the quality of the instruction (we did take some account of perceived course rigor in determining student assessment of instructors and courses).

There were some interesting regional surprises. In the State of North Carolina, Duke ranked third –behind both the University of North Carolina at Chapel Hill and Wake Forest. Northwestern was the top school in the Midwest, beating such regional powerhouses as the University of Chicago, Notre Dame, and, easily, Washington University in St. Louis. The South did extremely well among public national research universities, capturing five of the top 10 ranks, compared with one in the North East (SUNY Binghamton), two in the Midwest (University of Illinois, University of Michigan) and two in California (Berkeley and UCLA). The Big Ten Athletic Conference probably was the premier academic conference in the country after the Ivy League, placing seven schools in the top 50 public universities and a eighth, Northwestern, that ranked sixth among all universities.

Some other schools doing much better than in the US News rankings include: Samford, George Fox, Pacific and Biola universities, the University of Mississippi, and Westminster, St. John’s (Maryland), Hampden-Sydney, Doane, Hampshire, and Knox colleges. Some other schools doing worse than in the US News rankings include New York, Purdue and Carnegie Mellon universities, the University of Southern California, and Grinnell, Davidson, Occidental, Rhodes, Gettysburg and Carleton colleges.

How were schools selected? A majority of schools were in the top three tiers of the 2007 US News & World Report rankings of liberal arts colleges and national research universities. We added 121 schools offering masters or bachelor’s degrees termed regional institutions by US News. We took a few schools that were not in the US News rankings (e.g, Sarah Lawrence College), perhaps because of their failure to provide data (all of our data were obtained from external sources). And we selected the 50 largest schools that otherwise were unranked. To be sure, there are still hundreds of institutions we did not select.

We believe our rankings offer good information to those making college choices, and further think the emphasis on student instructional satisfaction and post-graduate success should please most potential future students and their parents. At the same time, we readily acknowledge data limitations. We need better data on post-graduate vocational success, and earnings data collected by payscale.com is a great potential future source, but is currently available for only some of our schools. Likewise, I would hope colleges would start voluntarily publishing data from sources such as the National Survey of Student Engagement, as well as information on the “value added” to learning over the college career from instruments like the Collegiate Learning Assessment. Still, this should assist those needing a good assessment of colleges and universities.

5 comments:

capeman said...

I can't imagine anything dumber than basing college rankings on the trash at ratemyprofessor.com!

davidjhemmer said...

I always throw out those "Who's Who" solicitations, I figured it was just as cam because they call you up and try to pressure you into buying a copy once they put you in it for "free".

http://rightwringprofessor.blogspot.com/

capeman said...

I used to get mail solicitations from "Who's Who".

Now I keep getting email from "Who's Who" telling me they just need more info (sample below). I send all this to the trash pronto. Perhaps I'm missing out on a great honor and the chance to move my university up in the Forbes rankings. (Very few of my students have honored me by going to ratemyprofessor.com, I'm proud to say.)


Dear Candidate,

Your response is kindly requested. You were recently appointed as a biographical candidate to represent your industry in the Who's Who Among Executives and Professionals, and for inclusion into the upcoming 2008-2009 "Honors Edition" of the registry.

We are pleased to inform you that on July 9th, your candidacy was approved. Your confirmation for inclusion will be effective within five business days, pending our receipt of the enclosed application.

The Office of the Managing Director appoints individuals based on a candidate's current position, and usually with information obtained from researched executive and professional listings. The director thinks you may make an interesting biographical subject, as individual achievement is what Who's Who is all about. Upon final confirmation you will be listed among thousands of accomplished individuals in the Who's Who Registry. There is no cost to be included.

We do require additional information to complete the selection process and kindly ask that you access this form on our website by Clicking Here

Unknown said...

What a joke.

Gee, the radical, embittered professor who advocates doing away with all federal and state support of higher education comes up with a college ranking that completely trashes public universities--except his beloved military academies.

What a surprise.

Vedder's tactics are eerily similar to those of PETA. He uses a few issues that the public can identify with (dog fighting...fur...college affordability...student loan debt) to gain legitimacy for a radical agenda that 99% of the public would dismiss out of hand were he to put it front and center (government mandated veganism...an end to all state and federal support for public universities...criminalizing pet ownership)

capeman said...

Lenny, of course this whole project is absurd. Yeah, Princeton has higher graduation rate, student satisfaction, alumni distinction, probably even better student debt than the University of Kansas, even UC Berkeley. Go ahead and trash Kansas, they must be doing a lousy job!

It's too bad that Forbes, which I once respected, is participating in this shoddy venture.

The information isn't completely worthless -- graduation rates, for example -- but you can find that in U.S. News.

But what does the rest of it tell you? Princeton graduates win more distinctions than Kansas graduates. Well, I should hope so.

Kansas takes in thousands of lower quality students, gives them a chance to get a low-cost degree (at which they fail at a higher rate), gives them an education at a small fraction of what is spent on a Princeton student, or probably a Wabash student -- and an opportunity to whine at ratemyprofessors.com?

I'd say there's a lot to admire there at Kansas.