Adam Cox sends along the following guest post:

The NAMUDNO argument did not go well for defenders of the Act. As Heather Gerken, Ellen Katz, and others have pointed out, the Justices spent a tremendous amount of time asking for comparative evidence–that is, evidence that intentional discrimination is a more significant problem in covered than non-covered jurisdictions. Neal Katyal tried hard to resist this approach, arguing that the Court should ask only whether things are bad enough in Texas to warrant Section 5’s coverage there. But time and again Justice Kennedy said that he was worried that Congress didn’t do enough to compare voting practices in covered with non-covered areas.
The demand for comparative evidence reflects a basic mistake that several folks have pointed out on the listserv. If Texas doesn’t look so bad relative to other states, there’s no way to know whether that’s because discrimination isn’t a problem in Texas, or instead because discrimination is a problem but Section 5 is an effective deterrent.
Even if we put aside that problem, however, both the Court and the lawyers repeatedly got the comparative evidence plain wrong during argument. More than half a dozen times, data from Section 2 lawsuits were cited as comparative evidence showing that discrimination is worse in covered than uncovered jurisdictions. Counsel for the intervenors said that these lawsuits were the “best evidence” that things were worse in the covered jurisdictions, and Justices Souter, Breyer, and Ginsburg relied repeatedly on these data as important empirical support for the Act. The problem is that the data from these lawsuits simply do not support the conclusion that discrimination is worse in covered than uncovered jurisdictions.
The Section 2 lawsuit data discussed during the argument were part of a report submitted to Congress. That report argued that the data demonstrated higher levels of discrimination in the south because, from 1982 to 2005, the success rate of section 2 suits in covered jurisdictions was higher than the success rate of litigation outside covered areas. But as Tom Miles and I have explained here on Rick’s blog, in a Columbia Law Review article, and in a colloquy with Ellen Katz, who authored the original report, the data do not support any comparative claims about the South.
I won’t retrace all of our discussion here, but I think it’s important just to note the three central reasons why the data can’t do the work that the lawyers or the Court might want them to do. First, while a summary comparison of lawsuits in covered and uncovered jurisdictions does suggest a disparity in success rates across covered and uncovered jurisdictions, this difference disappears when we run regressions to control for factors other than coverage itself. In other words, once we control for other variables (including, importantly, the identity of the judges), there is no statistically significant difference between success rates in covered and uncovered jurisdictions.
Second, and most fundamentally, it is a mistake to interpret differences in litigation success rates across jurisdictions as evidence of differences in the underlying levels of discrimination. The central methodological difficulty with drawing inferences about the extent of discrimination from litigated Section 2 cases is that the sample of cases is almost surely not representative of the entire class of voting rights claims. This is a problem that is so pervasive that it has a label among empirical scholars–it is known as the “Priest-Klein” problem. And fancy monikers aside, it is the intuitive reason why no one thinks that the success rates in tort lawsuits in Illinois are a solid source of information about the level of tortious conduct in the state, and why no one thinks that conviction rates in criminal cases in New York are a meaningful measure of the state’s crime level.
Third, even if we ignore these selection problems, the data quoted at oral argument are problematic because they mask a trend. The Section 2 study surveyed a two decade period and identified a difference between covered and uncovered jurisdictions over that period. But an average difference over two decades does not fit the conclusion that a difference justifying Section 5’s reauthorization remains today. And, in fact, the Section 2 data reveal that success rates in covered and uncovered jurisdictions have converged over time. In the 1980s and early 1990s, courts in covered jurisdictions were indeed more likely to find liability than courts in uncovered areas. But from 1994 to the end of the study in 2004, plaintiff success rates were nearly identical in covered and uncovered areas.
Tom and I do not believe that the convergence of success rates in recent years means that discrimination is no worse today in covered than uncovered jurisdictions. As we have emphasized repeatedly in our work, the success rates in Section 2 litigation are simply not a good measure of discrimination. Accordingly, our analysis should not be used to suggest that things are no worse today in the south than anywhere else. Conversely, however, the data discussed during the argument are not evidence of the opposite; they cannot provide the comparative evidence that things are still worse in covered jurisdictions. Much as the Court might like them to, these data can’t help resolve the question of Section 5’s validity.

UPDATE: Ellen Katz sends along the following response:

Adam Cox’s post repeats a series of claims to which I have already repeatedly responded, both on this blog and elsewhere. See here , here, and here.