San Francisco ? Chief U.S. Judge Vaughn Walker has always been characterized as a conservative with libertarian leanings. But after he struck down California’s voter-approved gay marriage ban this week, he was accused by some of being something else entirely: a gay activist.

Rumors have circulated for months that Walker is gay, fueled by the blogosphere and a San Francisco Chronicle column that stated his sexual orientation was an “open secret” in legal and gay activism circles.

Walker himself hasn’t addressed the speculation, and he did not respond to a request for comment by The Associated Press on Thursday. Lawyers in the case, including those defending the ban, say the judge’s sexuality — gay or straight — was not an issue at trial, and they have no definitive plans for it to be a factor on appeal.

But that hasn’t stopped a public debate that exploded in the wake of the 66-year-old jurist’s Wednesday decision. Most of the chatter has come from opponents of same-sex marriage.

“Here we have an openly gay federal judge, according to the San Francisco Chronicle, substituting his views for those of the American people and of our Founding Fathers who, I promise you, would be shocked by courts that imagine they have the right to put gay marriage in our Constitution,” said Maggie Gallagher, chairwoman of The National Organization for Marriage, a group that helped fund the ban, known as Proposition 8.

In response, the Gay and Lesbian Victory Fund, a political action committee for gay candidates, launched an online petition accusing Gallagher’s group of “gay-baiting.”

But the debate raises the question: Why is sexuality different from other personal characteristics judges possess? Can a female judge rule on abortion issues? A black judge on civil rights?

“The evidence shows that, by every available metric, opposite-sex couples are not better than their same-sex counterparts; instead, as partners, parents and citizens, opposite-sex couples and same-sex couples are equal,” Walker wrote in his exacting, 136-page opinion.

Gerard Bradley, a law professor at the University of Notre Dame, published a Fox News column in the hours before Walker filed his opinion faulting the media for not forcing Walker to address his sexual orientation.

And Byran Fischer, issues director for the American Family Association, urged the group’s members to contact their congressional representatives about launching impeachment proceedings because Walker had not recused himself from a case in which “his own personal sexual proclivities utterly compromised his ability to make an impartial ruling.”

William G. Ross, an expert on judicial ethics and law professor at Samford University in Alabama, said that a judge’s sexual orientation has no more relevance to his or her ability to rule fairly on a case involving gay marriage than it would for a deeply religious judge or a judge who had been divorced multiple times.