Accreditation Reform

by Andrew Gillen

There has been a flurry of activity concerning accreditation reform, which is not exactly normal. As luck would have it, we recently released a study on the topic. One thing that didn't make it into the study is a section I wrote on a proposed replacement system. I've copied it below for anyone who's interested, though this is a rough draft (both in writing and idea formulation), and the footnotes didn't copy and paste over (just get in touch if you'd like a word version with proper documentation).

*****

To paraphrase Alan E. Guskin (who was paraphrasing Michael Hammer and James Champy in a different context), the problem is that we are entering a new century with an outdated accreditation system designed to the address the problems of the last century. Given the problems that have arisen as a result, “it should be clear to the accreditation and higher education communities that a new model for quality assurance is needed, if for no other reason than to forestall future federal intrusion.”

In light of this, we believe that a better system could be devised, and should be implemented. In the words of CHEA’s president Judith S. Eaton, we need a focused system – one that holds colleges “accountable for achieving results” but does “not dictate how those results are to be achieved.” With that in mind, we propose the following replacement for accreditation. Our recommended system differs from the current system in three fundamental ways. First, eligibility for federal funding would be based on certification rather than accreditation. Second, certification would be granted to degree programs and courses rather than institutions. And third, multiple paths to certification would be available.

The most important change is that the new focus would be on certification, rather than accreditation. As Russell S. Kitchner notes,

“the notion of accreditation -- that process of determining if an institution is adequately positioned to fulfill its mission effectively -- is confused with certification, which often is an attempt to measure the degree to which an entity is, in fact, fulfilling someone else’s mission effectively. This difference is more than a matter of semantics, and historically, only the former concept was the purview of accreditors”

Accreditation is currently being used to determine if colleges are fulfilling the government’s goals in providing public money. But accreditation is not primarily concerned with the government’s goals, and is therefore not doing an adequate job of ensuring that they are met. A new certification system based on explicit measurement of the extent to which colleges are achieving the public’s goals with taxpayer money is highly desirable.

Second, certification of eligibility for federal funding would be granted for specific degree programs and even individual courses rather than accreditation being granted to entire institutions. Certifying individual programs and courses would, to paraphrase Hutchins, make colleges more like a series of separate programs and research centers held together by a central heating system. Ideally, this would be done with objective measures of student learning or other relevant outcomes. But such measures will not always be available, which brings up the third fundamental change.

The third major change is to have multiple paths to certification. As noted before, there is no one completely satisfactory method of determining which institutions should be eligible for federal aid. Because of this, it is important to have escape hatches to guard against the danger that any one path is inappropriately blocked, or is inappropriate for a particular type of educational program.

We recommend at least two tracks for certification.

Track 1: Meet Government Determined Cutoffs on Publicly Disclosed Data

More information about colleges needs to be disclosed to the public. This is already done for some categories of data, such as enrollment and employee counts and distributions by race and sex. But much more potentially useful data is not disclosed, or is disclosed but not in a manner that allows for meaningful comparisons. This needs to change. There are a number of potentially useful types of data that could be used for determining program or course eligibility.

One category of data that would be extremely useful for public accountability purposes would be information on what happens to students in the job market after they graduate. This information should be collected and disseminated for every degree program. Data on a number of outcome measures, such as job placement rates, passage rates on licensing exams, salary distribution etc., should be publicly available for every program that is offered.

If a program meets some cut off on a combination of these data, its students would be eligible for federal aid. For instance, a primarily monetary cutoff could be set for vocational programs, with institutions where every dollar spent in tuition (and appropriations?) yielded more than X dollars in higher starting salary for graduates being eligible for federal funding.

Another type of data that could be used for determining eligibility is student learning outcomes. This data could come from a variety of sources, including assessments specifically designed to measure the value added of a program or course. Another source of this data could be licensing exams that are in widespread use in a field, such as the CPA and the bar exam. Value added contributions to passage rates on those exams could be used to determine eligibility. Law schools that increase their students’ chances of passing the bar by a sufficient amount could automatically be eligible for federal aid. Data from other longitudinal databases could also be used for certification purposes. For instance, in Louisiana, the Board of Regents was able to match student performance with the colleges of education that trained their teachers. This allowed them to assess which programs produced the most capable teachers. Top performing programs on measures such as these could receive automatic eligibility.

One important issue is how the cutoffs would be determined. Making appropriate determinations is crucially important and we are more likely to see sound decisions if they are made in a technocratic manner that is shielded from political pressures. Fortunately, we have a model of a successful independent federal agency that makes technocratic decisions, namely the Federal Reserve. The Federal Reserve is the country’s central bank, and is tasked with conducting monetary policy, an extremely technical task. Throughout history, central banks that were not independent routinely violated the principals of sound money management as they succumbed to political pressure to inflate. To avoid this, members of the Board of Governors are appointed for very long terms, which insulates them from political pressure and allows them to focus on long term considerations. We would favor the establishment of a similar institution for higher education to determine the appropriate terms of eligibility for those programs following the first path.

Track 2: Meet (Mostly Discipline Determined) Cutoffs for Learning Outcomes

A problem arises with the first path in that there are fields where there is no certification or licensing exam in use, and it is not appropriate to measure outcomes based primarily on job market data. While we favor the development of suitable exams to the extent possible, it is clear that this would not be appropriate for every field or course. Thus, alternative path(s) are needed.

Many feel that “the place for quality assurance should have remained… with specific program evaluation associations such as the American Medical Association licensing medical programs, the American Bar Association approving law schools, and other various, subject-specific associations and agencies certifying program integrity within their areas of expertise.” We largely agree, and the second track in our proposed system would involve individual programs and courses receiving certification from a recognized entity. To offer degrees in chemical engineering for example, a degree program would need to be certified by an organization recognized to certify chemical engineering programs (note that there can be and should be more than one recognized entity to guard against monopolistic behavior of certifiers).

The basic idea is to utilize field determined standards to the greatest extent possible. It is important to note that the standards in a field need to be determined by private experts in that field, not public bureaucrats. An appropriate model is the American National Standards Institute (ANSI), described by A. Lee Fritschler as

“a private organization that is supported and managed by industry itself… ANSI accreditors’ standards become law when governments license professionals, e.g., in areas from health to plumbing. But it is the private organization which, through self-regulation, defines the standards government enforces. The standards are the product of the industry that operates in the field.”

To be clear, the government would not license graduates, but would be licensing certifiers. The certifiers would in turn certify degree programs and courses for federal funding purposes based on their students’ performance on field determined certification measures. Of course, this raises the issue of how the certifiers would be recognized. It is likely that the recognition process for accreditation could serve as a model (subject to stricter regulation as will be discussed shortly), though if more independent decisions are desired, an agency based on the Federal Reserve model could be established here as well.

What effects would it have?

In determining the effects that a program and course based certification system would likely have, the devil is in the details. While we will not attempt to lay out all of the specific details, we have identified three potential problems that deserve special attention. The first is that like the current system, such an arrangement could be subject to serious conflicts of interest. Second, such entities could engage in guild like behavior, such as artificially limiting the number of students entering a profession to drive up wages when an exam is required for licensing purposes. Third, as we’ve already seen, left to their own devices, specialized accreditors (the closest analogy to certifiers) have a tendency to rely on measures of inputs to try and increase spending for their specific fields.

We believe that the establishment and enforcement of reasonable regulations could reduce these dangers to tolerable levels. For instance, certifiers could be barred from having a stake in institutions that offer educational services, and be required to have a significant share of outside board members (to guard against conflicts of interest). They could also be barred from using input and process requirements (to guard against irresponsible recommendations). In addition, they could be forbidden from requiring college attendance to take the exams (to guard against guild behavior). The results below presume that such regulations would be established and enforced.

Quality Improvement

By freeing accreditors from their quality assurance obligations, they would be able to focus more exclusively on the quality improvement mission. Moreover, because accreditation would no longer be a near necessity, accreditors could specialize to a greater extent, which would have the potential to allow them to study particular issues and give more useful advice and guidance. Thus, we believe that our proposed system would improve the quality improvement role of accreditation.

Quality Assurance

The dual track certification of colleges for federal funding purposes would also improve all aspects of quality assurance.

To the extent that there are measures of quality under the current system, they are inappropriate, focusing primarily on inputs and processes that are thought to be necessary and sufficient to guarantee an adequate education. They are neither. By focusing on actual measures of the value added education provided by a program, the new system would see a vast improvement in the definition of appropriate measures of quality.

Similarly, the certification of minimum quality would see vast improvement. No longer would sufficient educational quality be assumed based on a college following the dominant formula in terms of inputs and processes. Instead, programs would be judged based on what their students have learned and what they can do.

The public would also have access to much better information and data about colleges. For the dual track certification system to work, data on learning outcomes and/or employment outcomes for each degree program needs to be publicly disclosed. This will allow for extremely useful types of additional information to be determined. One example would be that student-college specific estimations could be made. For instance, using the newly available databases, websites would emerge that informed a student who scored 150 on the LSAT that they would have a 20% higher chance of passing the bar if they went to Law School X instead of Law School Y. This would be very valuable information for students to have, and would likely lead to better matching of students and schools. Another example of the useful information this would provide would be rankings that are based on value added educational output rather than reputation and resources.

Health and Efficiency

The new system would also increase institutional autonomy. Accreditors are currently able to infringe on institutional autonomy because they have regulatory-like power over their member colleges. Some accreditors have decided to use this power to dictate practices that have no impact on educational quality, while others have tried to impose their own priorities on institutions. Under the new system these and other practices that limit the freedom of colleges would cease. For those colleges on the first track, some would not have any interference from third parties at all, with their eligibility for aid money determined entirely on labor market or certification exam outcomes. Even on the second track, programs would be evaluated based on outcomes only, giving the programs complete freedom of action.

Diversity of institution types and missions would likely remain mostly unchanged. To be sure, in some sense, diversity would decline if the data showed that certain educational practices were superior, and colleges flocked to adopt them (this would be good even if it reduced diversity). However, two things would help mitigate against even this reducing diversity. First, because degree programs would be certified, rather than each institution, there would be much more diversity in the measure of quality. Currently, there is a one size fits all approach, based largely on input usage. That would change under the new system, as some institutions would be judged on their ability to increase the salaries of their graduates, and others would be judged on learning outcomes defined by the field. These different measures of quality would yield very different programs. The second reason we probably wouldn’t see a decline in institutional diversity is that it is likely that different practices work better in certain fields, and programs would have an incentive to find and adopt the most appropriate practices in each case.

Innovation by new and existing colleges would likewise be encouraged. The current accreditation process does not measure the ends, and because of this, they feel the need to impose restrictions on the means used to achieve ambiguous ends. Colleges are more or less free to aim for whatever they want, but must achieve it using a one size fits all recipe. Placing such restrictions on the inputs that colleges can use and the way in which they can be used suppresses innovation. The new system, by changing the focus of evaluation from means to ends, would leave colleges completely free to experiment with new ways of providing an education. Existing colleges would have an incentive to continually reinvent themselves, and new colleges would have the opportunity to demonstrate their effectiveness without first having to conform to the existing paradigm.

There are two main downsides to the proposed system. The first is that some types of innovation would face more obstacles. Similar to the problems with a qualifications framework or certification schemes, entirely new types of degrees as well as new multidisciplinary degrees would have more difficulty in being approved. Under the current system, once an institution has accreditation, they are largely free to concoct new and/or multidisciplinary degrees. But when approval is based at the lower program or course level, these ventures will face increased scrutiny.

The second downside is that direct costs would be higher. Of course, it is easy to keep the costs of the current system low because it does not actually provide quality assurance. Any system that does provide quality assurance would cost more, and this proposal is no exception. On the other hand, indirect costs would fall. Much of the indirect costs of the current system come from mandates to use various unnecessary and inappropriate inputs, and these would cease under the new system. No longer would regional accreditors be dictating maximum course loads, nor would specialized accreditors be able to insist on better facilities for departments in their field. Thus, overall, costs would likely decline.