Will Baude, writing over at SCOTUS, is skeptical of my methodology in my new draft, The Most Sarcastic Justice, determining that Justice Scalia has been viewed the most sarcastic Justice by far in at least the last 30 years. Writes Will:

[I]t seems to me that Hasen’s methodology reflects a serious circularity in observations of the tone of Supreme Court Justices. When observers call a Scalia opinion “sarcastic” (or “stinging” or “caustic” or “angry” or anything else), I am not always sure whether this assessment is derived from what is actually written down, or from what the observer expects the tone to be.

Hasen acknowledges this and refers to it as the “‘echo chamber’ effect,” which he says he “cannot eliminate.” It seems to me that one way to eliminate it would be to have a set of readers, preferably those unfamiliar with the personal reputations of any of the Justices, read a bunch of anonymized Supreme Court passages and then evaluate the tone of the passages. Maybe the result would still be the same — I wouldn’t be surprised — but it might also be much less dramatic.

Adam Liptak raised this echo chamber effect with me in his NYT Sidebar column.  I briefly address it in a footnote in the piece, but I’m thinking I will need to expand the discussion as I continue to work on the piece. Here’s Adam:

Another possible objection is that people repeat the conventional wisdom and reinforce stereotypes. Justice Scalia may be sarcastic in the way President Gerald R. Ford was said to have been clumsy and Vice President Dan Quayle dim.

In an interview, Professor Hasen said he had taken that possibility into account. “My control for that,” he said, “is that I live in the real world.”

I wanted to elaborate on this point a bit. Yes of course it would be better to have randomized people read these opinions and classify sarcasm when they see them. But that is not a feasible research strategy. So my check was this: I looked at many of the references in law journals to a Justice’s opinion being sarcastic or caustic, and the judgment usually seemed to me to be on the mark, as a regular user of the English language. It is really hard to argue with the idea that many, many of the quotes pointed to from Justice Scalia are nasty or ironic, and often aimed at his colleagues.  Here’s an excerpt from my piece which shows what I’m talking about (footnotes omitted):

Justice Scalia has called other Justices’ opinions or arguments which he has disagreed with “bizarre,” “[g]rotesque,” and “incoherent.”…Justice Scalia has remarked that “Seldom has an opinion of this Court rested so obviously upon nothing but the personal views of its Members.” In a civil rights case, he ended his dissent by stating that: “The irony is that these individuals—predominantly unknown, unaffluent, unorganized—suffer this injustice at the hands of a Court fond of thinking itself the champion of the politically impotent.” In a gender discrimination case, he wrote: “Today’s opinion is an inspiring demonstration of how thoroughly up-to-date and right-thinking we Justices are in matters pertaining to the sexes (or as the C would have it, the genders), and how sternly we disapprove the male chauvinist attitudes of our predecessors. The price to be paid for this display—a modest price, surely—is that most of the opinion is quite irrlevant to the case at hand.” In an abortion rights case he declared: “The emptiness of the ‘reasoned judgment’ that produced Roe is displayed in plain view by the fact that, after more than 19 years of effort by some of the brightest (and most determined) legal minds in the country, after more than 10 cases upholding abortion rights in this Court, and after dozens upon dozens of amicus briefs submitted in these and other cases, the best the Court can do to explain how it is that the word ‘liberty’ must be thought to include the right to destroy human fetuses is to rattle off a collection of adjectives that simply decorate a value judgment and conceal a political choice.” Finally, in a concurring opinion in a substantive due process case, Justice Scalia wrote: “Today’s opinion gives the lie to those cynics who claim that changes in thisCourt’s jurisprudence are attributable to changes in the Court’s membership. It proves that the changes are attributable to nothing but the passage of time (not much time, at that), plus application of the ancient maxim, ‘That was then, this is now.

So I’m pretty confident that what observers are calling sarcastic or caustic actually fit the bill.