Monday, February 11, 2008

A New Way To Evaluate Colleges

By Richard Vedder

Many people complain, correctly, that popular college rankings, especially that of US News & World Report (USNWR) are largely based on inputs into the process of producing educational services, not actual educational outcomes. The Spellings Commissions and others have called for "value added" measures of learning and indicators of student outcomes. The colleges, for the most part, are resisting. The solution: devising an externally generated measure of outcomes. Two variables that would work perfectly: data on the Social Security earnings history of recent (last 10-15 years) graduates, and indicators of distinction among graduates as measured by entries in Who's Who in America. We at CCAP do not have the ability to get the Social Security earnings history, but we can get Who's Who entries.

We have taken an unbiased sample of 5,207 entries from the 2008 edition of Who's Who. We will be discussing these results in a series of blogs and probably at least one report. This is the first of those findings. They are a consequent of many hours of laborious digging by my beloved Whiz Kids --especially Jim Coleman, Jonathan Robe, and Thomas Ruchti (with moral support from Senior Whiz Kid Matt Denhart).

The broadest statistic:

54.23% of entrants had undergraduate degrees from one of the top ranked USNWR national universities or liberal arts colleges --a listing of well over 200 schools.

38.49% have undergraduate degrees from unranked (in the top tier) universities and colleges.

7.28% have no undergraduate degree.

In short, graduation from a top tier school helps your chances of becoming vocationally successful, but almost as many such persons are NOT graduates of such schools.

57 schools had 15 or more entries in Who's Who among our sample. They are, in order:

1.Harvard -153
2.Yale -91
3.U. of California, Berkeley -82
4.U. of Michigan, Stanford U.-59
6. U. of Pennsylvania, Princeton U. - 58
8. Columbia U. -55
9. Cornell U. -54
10. U. of Illinois, Urbana -53
11. U. of Texas, Austin; U. of Wisconsin, Madison -51
13. Dartmouth College -43
14. UCLA, U. of Minnesota -39
16. Northwestern - 38
17. New York University -37
18. U. of Notre Dame - 34
19. Duke U. -32
20. M.I.T., Michigan State - 31
22. Brown University - 30
23. Ohio State, Syracuse -29
25. U. of Virginia, U. of Maryland -28
27. U. of Chicago - 27
28. Williams College, Penn State, U. of North Carolina - Chapel Hill, Purdue U.-26
32. U. of Florida, U. of Washington -25
34. Georgetown U. -24
35. Indiana U., Boston, U. George Washington U. -23
38. U. of Iowa, U. of Kansas -22
40. Johns Hopkins, U.S. Naval Academy -21
42. Oberlin College -20
43. Miami of Ohio -19
44. U. of Colorado, SUNY Buffalo, U. of Rochester, Fordham U. Amherst College -18
49. U. of Missouri, Columbia, Rutgers U. -17
51. Rensselaer Poly, Washington U. in Saint Louis, U. of Oklahoma, Texas A&M -16
55. Ohio University, U. of Tennessee, Washington and Lee -15

Based on this extensive (about 6 percent of all entries) sample, there are a few big deviations from expectations based on the USNWR lists (national universities and liberal arts colleges). Among the universities, for example, Notre Dame, Michigan and Illinois are seriously under ranked relative to Duke, the U. of Chicago, or Washington U. in St. Louis. Among liberal arts colleges, Oberlin is seriously under ranked relative to Amherst. More about this issue in future blogs.

While all eight Ivy League schools ranked in the top 25, the Big Ten conference did well too, getting six schools in that elite group It is interesting to note, for example, that Michigan State has a larger number than, say, Johns Hopkins. Of the top 25 schools 15 were private --but 10 publics made the list.

To be sure, there are problems with the index, the sample is less than 100 percent, and, most important we have not taken account of enrollment differences between schools. We will address that last point soon in a future blog.

2 comments:

Bill Haydon said...

Two other data sets that I would consider important would be:

1) The propensity of undergraduates to eventually earn a doctorate. I did see a rather comprehensive set of data on the subject taken over a ten year range (undergraduate degree received mid 80s through mid 90s) that was quite interesting

2) The ability of a school's undergraduates to gain admission to (and earn advanced degrees from) the nation's top professional colleges...say the top 20 law, business, medical and engineering colleges--use whatever ranking you want.

Bill Haydon said...

Also, I would think that Dr. Vedder would agree that a single issue of Who's Who does not an adequate data set make. I would venture that the last ten years of Who's Who would be necessary--leaving open the question of whether an individual having multiple entries should count once or once for each year that he was included--this number could be compared against say an average undergraduate enrollment figure for the previous 30 years and an average undergraduate tuition (adjusted for inflation) also taken over the previous 30 years to derive both how likely matriculation at a certain university leads to Who's Who inclusion and how likely a given $$ of tuition spent also leads to the likelihood of inclusion.

BTW, this is nothing new. An early stab at ranking universities was actually published pre-WWII using this exact methodology. Here are its findings.

From So You're Going to College by Clarence Lovejoy, 1940. Colleges were ranked based on a study of how often their graduates had appeared in Who's Who in America.

Institution Name, average number of alumni in Who's Who editions 1937-1940, Rank
HARVARD UNIVERSITY 1433 1
YALE UNIVERSITY 1006 2
PRINCETON UNIVERSITY 515 3
COLUMBIA UNIVERSITY IN THE CITY OF NEW YORK 461 4
UNIVERSITY OF MICHIGAN-ANN ARBOR 454 5
CORNELL UNIVERSITY-ENDOWED COLLEGES 413 6
UNIVERSITY OF WISCONSIN-MADISON 342 7
UNIVERSITY OF CALIFORNIA-BERKELEY 290 8
UNIVERSITY OF PENNSYLVANIA 288 9
UNIVERSITY OF CHICAGO 279 10
AMHERST COLLEGE 267 11
BROWN UNIVERSITY 246 12
DARTMOUTH COLLEGE 231 13
MASSACHUSETTS INSTITUTE OF TECHNOLOGY 229 14
US MILITARY ACADEMY 221 15
UNIVERSITY OF MINNESOTA 201 16
STANFORD UNIVERSITY 201 17
WILLIAMS COLLEGE 190 18
US NAVAL ACADEMY 189 19
UNIVERSITY OF ILLINOIS AT URBANA-CHAMPAIGN 186 20
INDIANA UNIVERSITY 181 21
JOHNS HOPKINS 172 22
OHIO STATE UNIVERSITY 169 23
UNIVERSITY OF NEBRASKA 162 24
WESLEYAN UNIVERSITY 161 25
UNIVERSITY OF MISSOURI 155 26
UNIVERSITY OF KANSAS 146 27
STATE UNIVERSITY OF IOWA 139 28
OBERLIN COLLEGE 137 29
CUNY COLLEGE OF THE CITY OF NEW YORK 129 30
OHIO WESLEYAN 120 31
UNIVERSITY OF VIRGINIA-MAIN CAMPUS 120 32
UNIVERSITY OF NORTH CAROLINA CHAPEL HILL 116 33
SYRACUSE UNIVERSITY 116 34
DE PAUW UNIVERSITY 115 35
NORTHWESTERN UNIVERSITY 113 36
BOWDOIN COLLEGE 110 37
VANDERBILT UNIVERSITY 101 38
THE UNIVERSITY OF TEXAS AT AUSTIN 92 39
CASE WESTERN RESERVE UNIVERSITY 95 40