I'm No Cherry Picker (But I Do Have An Agenda)

by Andrew Gillen

The plot thickens on efforts to discredit my new study, and like the last attempt, this latest one miraculously attempts to do so without mentioning anything specific to the study.

The Badger Herald’s Taylor Cox interviewed me yesterday, and she also spoke with someone who doesn’t like the report - Haley Chitty from the National Association of Student Financial Aid Administrators. According to the article, the NASFAA “disagrees with [my] methodology,” accuses me of being “selective in the evidence [I use]”, and thinks that CCAP “had already concluded federal aid leads to increases in tuition, and then they found facts that were convenient.”

Their concerns would carry more weight, if anything was offered to substantiate the claims either in the article or on their website, but I’ve been unable to find specifics anywhere. I am willing to give the benefit of the doubt to the NASFAA, stipulating that they are in fact familiar with the study and that the evidence to back up their claims just didn’t make it into the Badger Herald article or the NASFAA website. However, when someone says “we’ve seen this kind of report before and it’s usually someone with an agenda and not necessarily a very insightful report”, it raises the possibility that they are just attacking research they don’t like by using the classic technique of questioning the motivation/agenda of the author, making accusations about cherry picking evidence, and claiming there are problems with the methodology. These coincidentally, are the precise charges leveled at me.

Perhaps I will soon develop a thicker skin and be able to ignore such unsubstantiated criticism, but I guess I’m still too young and naïve, holding out hope that my ideas will be judged on their merits instead of by where I work. My task of responding is a bit difficult because without any specifics, their stance can basically be boiled down to we think he’s wrong. Since the No I’m Not defense is fairly unsatisfying, I’ll address the three broad charges in turn, even though no substantiation was given for any of them.

To be honest, I’m a little confused about the charge of cherry picking evidence. I focused on state appropriations, student loans, and federal grants, which together constitute the vast majority of government-provided financial aid. If cutting down most of the orchard constitutes cherry picking, then I guess they have a point.

As to the claim that CCAP has an agenda, we are definitely guilty as charged, as I presume is the NASFAA (unless of course they’re claiming to be agenda-less). Anyway, we do have an agenda and that agenda is to make college more affordable and more productive. Anything that my research indicates is an obstacle to those goals is fair game, including financial aid. The concern that I set out to attack financial aid is certainly misplaced. Why on earth would I advocate for Pell grants and modified student loans if I’m anti-financial aid?

As for problems with my methodology, without them indicating what those problems are, I really don’t know what to say. Do they have a problem with any methodology that reaches conclusions that they don’t like? Without knowing what their objections are, as opposed to the fact that they have objections, it’s hard to say. As it stands, I have already thoroughly debunked the only methodology related criticism I’ve seen so far - that I confused correlation with causation (see this post or comment 25 here). Personally, I still think that if I am wrong it is because my theory is wrong, as opposed to because I used an inappropriate methodology, but I certainly welcome thoughts on this from NASFAA or anyone else. Just try to make it a little more substantial than we think you’re wrong.

Near the end of the article, I’m chastised for not “letting the research speak for itself.” I would love to let my research speak for itself, but so far, my critics are talking about everything but the research.