It's Really Hard to Rate Colleges, the Department of Education Says

Harvard University campus in Cambridge, Mass. Photographer: Michael Moran/OTTO

For more than a year and a half, the Department of Education has been trying to figure out a way to rate colleges, in order, one day, to tie federal financial aid to whether schools are accessible, affordable, and provide good outcomes for students. Today the department put out what it called a “framework” for the plan, filled with questions and options the department still “is exploring.”

The “framework” falls short of what most people expected from the announcement—a more complete first draft of the plan. As Robert Kelchen, an assistant professor at Seton Hall University, says, the release makes clear the rating plan “is very much in its infancy.”

The department has taken great pains to listen to students, scholars, and schools and says it understands it needs to be careful about which measures to choose, lest it create perverse incentives for schools. It also must grapple with the fact that big gaps are in available data. The best source of dropout rates, for example, doesn’t account for students enrolled part-time or who have returned to school after previously dropping out.

The framework does lay out roughly a dozen metrics the department may use, a list as notable for what’s not included as for what is. Average student debt is not under consideration because, department officials told reporters, it could penalize colleges that admit lower-income students. Instead, the framework accounts for net price—the cost of attendance minus grants and scholarships—and the percent of student borrowers who make their loan payments on time as an indication of whether the debt students take on is manageable based on what they earn.

Under Secretary Ted Mitchell said that the department still hadn’t determined how the individual metrics will translate into an overall grade. The ratings will sort schools into broad groups: “high-performing, low-performing, and those in the middle.” That’s in line with what Deputy Under Secretary Jamienne Studley told me a year ago, when she said she liked how Cook’s Illustrated lays out “the key characteristics of blenders and canned tomatoes” and summarizes the results as “highly recommended,” “recommended,” “recommended with reservations,” and “not recommended.” According to Bloomberg News, the department still isn’t sure if it will have one overall summary rating for the school or if it will get designations for specific aspects, such as “affordability” and “access.”

The framework lists many unanswered questions: The department wants to give schools credit for improving but wants suggestions how to do so. For now it is proposing rating two-year schools against other two-year schools and rating four-year colleges with each other. But it is also figuring out if there are more “granular” way to compare peers. Officials said perhaps schools that have selective admission would have different expectations than those that are open access.

Comments on the framework are due Feb. 17, and the department official swears the department will have version 1.0 of the ratings out in time for the 2015-16 school year. They’ve got their homework cut out for them.

    Before it's here, it's on the Bloomberg Terminal.