The controversy over Supreme Court Justice Antonin Scalia’s hunting trips, especially the one he shared with Vice President Dick Cheney, has upset and saddened many lawyers and others concerned with the continuing legitimacy of the U.S. Supreme Court. In a number of ways, however, the public debate has been somewhat misplaced.

The specific accusations against Justice Scalia have centered on his having hunted with the Cheney during a period in which the Supreme Court is to hear a case involving a commission chaired by the vice president. A litigant in that case, the Sierra Club, asked that Justice Scalia recuse himself from the deliberations on the grounds that his socializing with the vice president might have influenced his judgement. After several weeks of controversy, Justice Scalia refused to do so.

Recusal is an important device used by litigants and judges to ensure not only that cases are decided fairly but also to ensure that there is no “appearance of impropriety.” It simply means that the judge recusing himself has decided not to hear or decide the case. It is common for litigants to ask for recusal and it is common for judges to decide to recuse themselves or to refuse to do so. Justice Scalia himself recently recused himself from another case because he had commented on its merits before it was heard, an act that put his impartiality in question.

Certainly, there can be no negative inference drawn when a judge recuses himself; to do so simply means that the judge has decided either that he cannot remain neutral or that his hearing the case would call his neutrality — and the decision’s fairness — into question.

Why not recuse himself?

Given this fact, why did Justice Scalia not simply decide to recuse himself in the Sierra Club case? It’s always dangerous to speculate on judicial motivation, particularly when a judge is as bright and complex as Justice Scalia, but in this case Scalia’s written opinion and comments on the requested recusal give us a fairly good basis on which to speculate. He states in this opinion that his refusal stems from two fundamental facts.

First, he was not influenced by the hunting trip with Cheney and, in fact, never discussed the case with the vice president. Second, if he were to recuse himself in this case then he and the other justices would have to recuse themselves in many cases or live like hermits, since they often socialize with government officials and lawyers in Washington, many of whom are often involved in cases before the Supreme Court.

I find it relatively easy to believe Justice Scalia on the first point. Even if one were to discount his comments that he had not spoken to the vice president about the case, it still seems unlikely that such a conversation would have influenced the justice. In fact, Justice Scalia’s views, both legal and political, are well known. Of all the present Supreme Court justices his decisions are the most predictable, and few commentators have any serious doubt that he will side with the government in this case and would have done so, hunting trip or not.

Possible Sierra Club motives

Indeed, one might argue that the main reason that the Sierra Club made its motion for recusal was precisely because it sees Justice Scalia as a likely vote against its case and wants him off the case. To the extent that the media and the public have been upset that this hunting trip may have influenced Justice Scalia as regards the merits of the case, therefore, it is most likely not a serious concern. I suspect that very little in the way of social interchange can affect Justice Scalia’s opinions; on the contrary these often seem set in stone.

It is the second ground for Justice Scalia’s decision not to recuse himself that I find less convincing and most troubling. A justice of the Supreme Court of the United States has a very different job than that of a trial judge in a small town. It would be rather unfair to tell a small town judge that she couldn’t socialize with potential litigants. To do so would condemn that judge to a very lonely life.

But justices of the U.S. Supreme Court are nine of the most powerful people in the United States. Their decisions can literally change the course of history. It is hard to believe that of all the millions of people in Washington they couldn’t choose friends who were not among the rich and powerful. Justice Souter, for instance, doesn’t appear to socialize much, if at all, with the Washington power elite.

What Justice Scalia seems unwilling to accept is that with the power he has been given comes great responsibility and great burdens. We can and should expect the justices to do everything they can not only to maintain their impartiality in fact but also to avoid even the hint of scandal or controversy, even if this means they cannot spend recreational time with the Washington elite. There are many lawyers and judges who would be willing to give up their entire social lives and live as hermits to sit on the Supreme Court of the United States.

Harming the court’s image

In the end, Justice Scalia’s actions involve more than the question of whether or not he has strictly adhered to the formal standards of legal ethics. Instead, his actions go to the far more important question of whether he has behaved in a manner that harms the legitimacy and public image of the U.S. Supreme Court. I believe that, in this case, he has done so and done so needlessly. It is difficult for me to feel great sympathy with Justice Scalia’s argument on this point.

When I was a dean, a job of infinitely less importance than that of a judge, let alone a Supreme Court justice, I was conscious of my institutional position every waking moment and did everything I could to avoid even the hint of scandal, even when that meant my life was restricted as a result. There’s an old Randy Newman song I think the justice needs to hear; one lyric goes “it’s lonely at the top.”

He can still go hunting, by the way. I’m sure that there are plenty of folks around the country who will never be litigants in his court who’d be glad to take him hunting. Indeed, he might enjoy a day out with a few ordinary people. It might do him — and the Supreme Court — some good.

Mike Hoeflich is a professor in the Kansas University law school.