Justin Grimmer for Politico Mag:

I’m a political scientist who develops and applies machine learning methods, like forecasts, to political problems. The truth is we don’t have nearly enough data to know whether these models are any good at making presidential prognostications. And the data we do have suggests these models may have real-world negative consequences in terms of driving down turnout.

Statistical models that aggregate polling data and use it to estimate the probability of each candidate winning an election have become extremely popular in recent years. Proponents claim they provide an unbiased projection of what will happen in November and serve as antidotes to the ad hoc predictions of talking-head political pundits. And of course, we all want to know who is going to win.

But the reality is there’s far less precision and far more punditry than forecasters admit…

In our paper, we show that even under best-case scenarios, determining whether one forecast is better calibrated than another can take 28 to 2,588 years. Focusing on accuracy — whether the candidate the model predicted to win actually wins — doesn’t lower the needed time either. Even focusing on state-level results doesn’t help much, because the results are highly correlated. Again, under best-case settings, determining whether one model is better than another at the state level can take at least 56 years — and in some cases would take more than 4,000 years’ worth of elections….

While we lack evidence that probabilistic forecasts are accurate, there is real evidence that they can create confusion and potentially deter voters from coming to the polls.

A large-scale survey experiment conducted by Westwood, New York University’s Solomon Messing and the University of Pennsylvania’s Yphtach Lelkes shows that forecasts are deeply confusing to Americans — causing them to mix up a candidate’s probability of winning with that candidate’s expected vote share.

In their experiment, they found that sometimes when people see a model forecast (say, a 58 percent chance of victory, or a 58 in 100 chance) they erroneously think that this means that a candidate will win 58 percent of the vote. Indeed, they write, “More than a third of people estimate a candidate’s likelihood of winning to be identical to her vote share, and on average people estimate that likelihood to be closer to the vote share than the probability of winning after they see both types of projections.”

These election forecasts may also create a false sense of security among some citizens about the odds of their side winning, which ultimately causes them not to vote because they feel it’s not necessary.