Professor takes a closer look at university rankings game

Every spring, not long after March Madness, there is another kind of tournament of interest to many Kansas University faculty, students and administrators. This is not a sports tournament by which the winners and losers of games are determined but a tournament by which some universities and university programs are claimed to be better than others: the annual rankings of colleges and universities and their degree programs by U.S. News & World Report and other organizations.

The preliminary 2011 U.S. News & World Report rankings have just been released, indicating that the KU city management and urban policy graduate degree program is ranked No. 1 overall (among both public and private universities) and the special education graduate degree program is ranked No. 2 overall. Among public universities, there are 39 KU graduate and professional degree programs that are ranked in the top 40 nationally. The preliminary rankings for 2011 are very nearly the same as the 2010 rankings. Last year, not long after the announcement of the 2010 rankings, there were calls by top university and state officials for plans and resources dedicated to improving KU’s rankings.

So-called third-party (such as U.S. News & World Report) rankings have been around long enough now so they have been subjected to extensive analysis, and we now know a lot about them.

First, rankings privilege longevity. It is no wonder that many of the oldest universities in American are routinely ranked at the top. At KU, our two highest-ranked programs have been building their reputations over many years, about 60 years in the case of city management and urban policy, and 50 years in the case of special education. Highly ranked programs take many years to build.

Second, over the longer run, say more than two or three years, university and degree program rankings tend to equilibrium, or to be stable. In a particular year, a university or a degree program may go up or down one or two or even more ranks. Over longer increments of time, however, university and degree programs will tend to revert to their previous positions. Short-run changes in rankings are often the result of changed evaluation criteria by those doing the rankings or other “noise” in the evaluation process. So, when U.S. News & World Report presents it annual rankings as “news,” some cynics familiar with the rankings process refer to it as “noise.”

Third, specialization matters. One of the dominant characteristics of universities consistently ranked in the top 20 is that they have many disciplines and degree programs that are also ranked in the top 20. To be generally good, a university must be specifically good.

Fourth, to retain their ranks, universities in the top ranks will invest more in their top-ranked programs. Universities in the intermediate ranks will also invest more in the better-ranked programs as they attempt to improve both specific and overall ranks. And so, as KU strengthens particular programs in pursuit of better rankings we can be sure that other universities are also strengthening theirs. It may be that KU will need to strengthen its ranked programs just to retain its present rankings in those programs. This has been described as a kind of academic arms race.

Fifth, U.S. News & World Reports’ overall university rankings are based on debatable criteria. The critics argue that these criteria merely mirror the characteristics of elite colleges and universities, reflect prestige and status more than actual measures of quality, and are mostly a function of three characteristics, fame, wealth, and exclusivity. Supporters invite critics to develop ranking systems with their preferred criteria. When this has been done, the top universities are about the same but often in a different order, but the universities in the intermediate rankings are rather different than the U.S. News and World Report rankings.

Sixth, it is argued that university- and degree-ranking regimes are the enemy of college and university creativity and innovation.

Finally, if the 2011 U.S. News and World Report rankings again catch the interest of top university and state officials, I suggest an open forum or workshop involving interested and informed stakeholders on the subject of how or whether to respond to the most recent rankings.

— H. George Frederickson is the Edwin O. Stene Distinguished Professor of Public Administration at Kansas University.