What I Was Wrong About This Year

I’ve come to realize that I was wrong about a major aspect of probabilities.

They are inherently hard to grasp. That’s especially true for an individual event, like a war or election. People understand that if they roll a die 100 times, they will get some 1’s. But when they see a probability for one event, they tend to think: Is this going to happen or not?

They then effectively round to 0 or to 100 percent. That’s what the Israeli official did. It’s also what many Americans did when they heard Hillary Clinton had a 72 percent or 85 percent chance of winning. It’s what football fans did in the Super Bowl when the Atlanta Falcons had a 99 percent chance of victory.

And when the unlikely happens, people scream: The probabilities were wrong!

Usually, they were not wrong. The screamers were wrong.

It’s not enough to say an event has a 10 percent probability. People need a story that forces them to visualize the unlikely event — so they don’t round 10 to zero.

Imagine that a forecast giving Candidate X a 10 percent chance included a prominent link, “How X wins.” It would explain how the polling could be off and include a winning map for X. It would all but shout: This really may happen.

David Leonhardt, New York Times, December 24, 2017

Trump won, and Northam crushed Gillespie. Why believe polls ever again?

Back in 1962, Elmo Roper, a pioneer in public opinion polling, identified a problem in his field.
“A preference for certainty over doubt, for the plausible over the proved, for drama over accuracy, for hunch and intuition over the hard-to-assemble facts, is a common human tendency,” he

Fifty-five years later, Nate Silver — today’s political-statistics guru — has the same complaint.

“There’s a strong desire for a narrative, and a lot of groupthink,” the founder of Fivethirtyeight.com told me last week.

Silver, not for the first time, argues that the numbers themselves are not to blame. In aggregate, and allowing for margins of error, they were reasonably accurate.

It’s the interpretation by journalists — particularly the pundit class — that’s to blame, he says. For one thing, they often don’t use the aggregation of the various polls, but rather a single one, often the most recent.

And they interpret them to create a dramatic — preferably surprising — horse-race story.

“They start out with an idea and backfill the justification,” using whichever numbers help make the case, Silver complains.

Margaret Sullivan, Washington Post, November 12, 2017

The Polling Crisis: How to Tell What People Really Think

Until as recently as ten years ago, polling organizations were able to tap into public opinion simply by calling people at home. But large segments of the population in developed countries have given up their landlines for mobile phones. That is making them more difficult for pollsters to reach because people will often not answer calls from unfamiliar numbers.

So the pollsters are fighting back. They are fine-tuning their efforts in reaching mobile phones, using statistical tools to correct for biases and turning to online surveys. The increasing number of online polls has prompted the formation of polling aggregates, such as FiveThirtyEight, RealClearPolitics and Huffington Post, which combine and average the results to develop more nuanced forecasts.

“Polling’s going through a series of transitions. It’s more difficult to do now,” says Cliff Zukin, a political scientist at Rutgers University in New Brunswick, New Jersey. “The paradigm we’ve used since the 1960s has broken down and we’re evolving a new one to replace it—but we’re not there yet.”

Ramin Skibba, Scientific American, October 19, 2016

The Social Science Research Behind Political Campaign Ads

U.S. presidential candidates advertise in battleground states to increase voter turnout. But a new study says ads also have a big impact on campaign contributions.

GREENE: Are you saying there might be untapped money in some of these places and if they run ads, people might say – oh, I’ve liked this candidate anyway. But now that you’re actually making the effort, I might open my checkbook?

VEDANTAM: Exactly. There’s also a psychological reason. If you live in a safe red or a safe blue state, you have the feeling that you can’t do much to influence the election because the outcome in your state is a foregone conclusion. But one way to get these voters involved is to get them to pull out their checkbooks and credit cards so they can say, you know, my vote might not matter to the outcome, but maybe my $200 can make a difference.

[A]dvertising in Democratic states would require campaign managers to take the kind of risks that campaign managers don’t like to take.

NIEBLER: It doesn’t pass the gut check to a lot of campaign managers, I think. Someone can easily look and say, you spent $100,000 advertising in San Francisco, when they maybe lose a state by a half a percent or even a couple of hundred votes that they could have won maybe having spent a little bit more money there.

VEDANTAM: The bottom line, David, is that campaigns actually tend to be quite risk-averse. A campaign manager who takes this kind of gamble and loses will probably never be hired to run another campaign again.

Shankar Vedantam, NPR, October 28, 2016

Poll: Most see a Hillary Clinton victory and a fair count ahead

Why Are the Different Presidential Forecasts So Far Apart?

Hillary Clinton currently has a 71 percent chance of winning the presidency, according to The Upshot’s forecasting model. This is down from 90 percent last month, but higher than some other models we’re tracking, which put the odds between 58 percent and 85 percent.

Part of the discrepancy comes from the use of different information. The PredictWise number — 74 percent — incorporates a sharp jump in betting markets that occurred during the first presidential debate. This jump, if it’s real, is not yet reflected in polls, which take days to conduct.

Josh Katz, New York Times, September 29, 2016

When You Hear the Margin of Error Is Plus or Minus 3 Percent, Think 7 Instead

In a new paper with Andrew Gelman and Houshmand Shirani-Mehr, we examined 4,221 late-campaign polls — every public poll we could find — for 608 state-level presidential, Senate and governor’s races between 1998 and 2014. Comparing those polls’ results with actual electoral results, we find the historical margin of error is plus or minus six to seven percentage points. (Yes, that’s an error range of 12 to 14 points, not the typically reported 6 or 7.)

David Rothschild and Sharad Goel, New York Times, October 5, 2016

Links to Information on 2016 Presidential Polls
















Here’s Why HuffPost Is Dropping Polls That Rely Only On Landlines

As more Americans have made a mobile phone their main phone, polls that contact only landline phones look increasingly behind the times. HuffPost Pollster has decided we need to evolve. 

By the end of last year, 48 percent of American adults didn’t have a landline phone. Another 16 percent had a landline but relied mostly on their mobile phone to make and receive calls.

That’s 64 percent of Americans who are only or primarily reachable via mobile phone.

In a higher-turnout contest with more diverse voters, however, there’s reason to think landline-only surveys can no longer accurately measure opinion.

Natalie Jackson, Huffington Post, August 1, 2016

Why the Surprise Over ‘Brexit’? Don’t Blame the Polls

In a sense, the E.U. referendum joins a pretty long list of election forecasting errors. But this one was a bit different: It was not a cataclysmic polling failure.

The polls consistently indicated that there was a very real chance that Britain would vote to leave. Polling averages even showed “Leave” with a lead for most of the last month; over all, 17 of the 35 surveys conducted in June showed the Leave side with the edge, while just 15 showed Remain ahead.

Yet at the same time, betting markets indicated that Remain was a clear favorite. The arguments for making Remain a favorite were understandable, but in retrospect, some look more like wishful thinking than a fair-minded assessment of the data.

Nate Cohn, New York Times, June 24, 2016