A real critique

by Andrew Gillen

Reuben Ternes has read my financial aid study, and provided a real critique, which I appreciate. He brings up a number of points that I thought I’d respond to.

[CCAP] “seems to have already decided that education costs too much.”

This is largely true, but I really don't see the problem. CCAP was started after Rich wrote Going Broke by Degree: Why College Costs Too Much, which among other things concluded, as the subtitle suggests, that, well, college costs too much. The organization was started to try and do something about that. The book came out in 2004, and CCAP started in 2006, which means that the organization and our mission is an outgrowth of a disinterested analysis. Perhaps I’m missing something here, but I really don’t see how that undermines our ethos. Should we rewrite the book every time we put out a paper?

BTW, since Rich was a semi-retired professor when he wrote the book (and still is), shouldn’t his motivation have been pro higher ed establishment? To the motivation obsessed, shouldn’t that give us even more credibility? I don’t put a lot of weight behind that line of reasoning, but those who try to discredit our work by pointing to nefarious motivations sort of have to if they want to be at all logically consistent, don't they?

I understand conceptually why people are so focused on motivations, but I don’t share that particular focus – I’d rather just judge ideas on their merits rather than on who made them.

“I still can't get over the fact that the author seems to think all you need to provide causation is a correlation and a good theory”

The blog post he is referring to was in response to other people claiming that I was confusing correlation with causation, which was not the case. I basically pointed out that the claim of causation came not from the correlations, but from the theory that predicted/explained the correlations. I don’t think I need to emphasize that that is not the same as saying all you need is any old theory and a high R^2, but perhaps an example will illustrate. Justin Fox recently had a post where he mentioned an old David Leinweber paper showing that “butter production in Bangladesh, U.S. cheese production, and sheep population in Bangladesh and the U.S. together ‘explained’ (in a statistical sense) 99% of the annual movements of the S&P 500 between 1983 and 1993.” That’s an amazing correlation, but even if there was an accompanying theory nobody would take it very seriously. The fact that there are better theories/explanations, even though their R^2 isn’t as high, keeps us from walking around with the notion that sheep, butter, and cheese determine the level of the S&P (or vice versa).

This is why social sciences have a heavy dose of art - the weighing of multiple competing theories, none of which perfectly matches the data. This is what I attempted to do in the second half of the paper, by showing numerous areas where my theory matched the data better than the old theory.

“my biggest complaint about the article is its scope”

Any analysis that tries to generalize about higher education is open to this criticism. There are over 4k degree granting institutions in the US, with varied goals, circumstances, and opportunities. The fact that no one model will be applicable to all of them does not seem to me a valid reason for not looking for generalizations or rules of thumb. I would think someone whose blog is titled “A Patterned World” would see the value in that. Having said that, I shouldn't dismiss his point, since it doesn't hurt to remind us policy types that care is needed when drawing generalizations.