The national exit polls were off by 6% points (predicting Kerry with 51% to Bush’s 48% and not the reverse). What happened to make these so wildly inaccurate? They were obviously off for Ohio and Florida as well, by 4 points each. I know that Voter News Service was phased out after mistakes in 2000. What happened now?
UPDATE: Michael McDonald of George Mason/Brookings writes (in a revised version of some comments sent to the election law listserv):
- Having just arrived home from working the night at the exit poll service (Edison-Mitofsky), perhaps I can shed some light on this issue. First, it
was clear to us in the mid-afternoon that the exit polls were over-sampling
women, which was producing a pro-Democratic bias in the sample. Corrective
measures were taken, and we alerted our subscribers well before the polls
closed that the exit polls were showing the bias, which delayed clear Kerry
calls in states like PA and NH. Perhaps you heard the phrase “not enough
information” throughout the night, as we were waiting for actual vote
returns to confirm the exit polls.
How the exit polls are conducted, how they are weighted, and how they are
used by the networks on election night go well beyond simple sampling
methodology. Let’s just examine the actual calling of the election. If a race is outside the margin of error AND it comports with
the best previous guess of the election outcome AND it comports with
previous vote shares for various offices in the sample precincts, then we
are able to call a race immediately when the polls close. If not, a
combination of exit poll results, past election results, and actual election
returns are used to make calls as data come in from the Associated Press.
The system worked as it should, and in no case did we make a call this
election that later had to be reversed.
The only issue was a delay in getting the calls out for a few states that
might have been called sooner if we had more confidence in the exit polls.
I think we can all agree it is more important to get it right than get it
quickly, and in some cases we held back calls even longer because we weren’t
certain that something funny was going on with the data, like it did in FL
in 2000. In PA, for example, the Philly suburbs were late in reporting so
we didn’t want to make a call without some hard numbers from those areas to
confirm our exit polling. For states like FL and OH this time, that were
close, there was not enough information in the exit polls to make a call in
those states when the polls closed, so we were going to have to wait for the
raw numbers came in anyway.
The real problem with the exit polls is that everybody wants to know the
results before the polls close. Once the first wave of polls went out to
the subscribers, it took about a half hour for our polls (and some fake
polls) to make it onto the internet. Warren Mitofsky continually warns
everyone not to trust the early exit polls because they are what they
are…polls, and they are subject to error (not just sampling error). Too
bad more people don’t appreciate this point.