How has election polling accuracy fared in 2017 and 2018?

Posted by

Donald Trump’s victory over Hillary Clinton in the 2016 presidential election defied the predictions of most well-known pollsters, who had estimated Clinton’s likelihood of winning in the 70 to 99 percent range. After Trump’s win, many headlines suggested that the polls had failed, and the conservative-leaning pollster Rasmussen found that only about a quarter of likely U.S. voters trusted most political polls (although this was based on a poll, too).

The reality, however, is more complex. On one hand, state-level polls made mistakes: in the Electoral College contest, state-level polls showed Clinton with a slim advantage, but they drastically underestimated support for Trump in the Upper Midwest swing states of Wisconsin, Pennsylvania, and Michigan. State-level polls also missed a late swing to Trump among undecided voters—many of whom settled on a candidate just a week before the election—and failed to adjust for over-representation of college graduates, who lean Democratic and are more likely to take polls. As a result, state-level polls were fairly poor (although polling errors were still within the normal range, at only 0.4 percentage points worse than the average since 1972).

But while there were issues on the state level, national polls had a good year by historical standards: 2016 national polls ranked among the most accurate in popular vote estimation since 1936. They indicated that Clinton had about a 3-percentage-point lead, and they were on the mark. She won the popular vote by 2 percentage points.

Moreover, research has debunked accusations of consistent partisan polling bias. While it’s true that polls’ statistical bias varies by election, it shifts unpredictably; polling errors can occur in both directions and are almost impossible to predict. Over the long run, U.S. election polls have exhibited very little bias toward either party.

But how have the polls fared more recently in the U.S.? Believe it or not, better than usual. Polls of elections since 2016—that is, the 2017 gubernatorial elections and the 2017 and 2018 congressional special elections—have been slightly more accurate than average. Consider the nine House, Senate, and gubernatorial elections with polling in 2017 and 2018. Among the six special House elections with polling in the three weeks before voting day (AZ-08, GA-06, MT-at-large, PA-18 and SC-05), the average absolute error per poll was 4 percentage points: 1.1 points better than the average poll of 5.1 points in House specials from 2004 to 2016 (the lower the error, the stronger the poll).

Meanwhile, polling for the Alabama U.S. Senate and the New Jersey and Virginia governors’ races fared about average. Of the 31 polls conducted in the gubernatorial contests, the average error was 5.2 points, only slightly higher than the 5.1-point average from 1998 to 2015. In Alabama’s race, the average error for the 16 polls in the last three weeks was a loftier 5.9 percentage points, but don’t be fooled: from 1998 to 2016, the average error for Senate elections not held on Election Day of a midterm of a presidential year was 5.8 percentage points—on par with the Alabama race polls.

In short, the 2016 presidential election certainly didn’t spell doom for pollsters in 2018; as political scientist Christopher Wlezien said, “the sky is not falling” for polling accuracy.

Nevertheless, there are lessons to keep in mind for the November midterms. State-level polling does err more frequently than national polling, so be cautious about overinterpreting a candidate’s lead in U.S. Senate and gubernatorial polls. Moreover, because statistical bias is unpredictable, neither Democrats nor Republicans should count on categorically outperforming their polls. Most importantly, voters must be aware of polling’s inherent limits. Good polls pick the winner correctly an impressive 80 percent of the time, but they can’t always predict the future. With the midterms around the corner, people ought to think realistically about how precise—or imprecise—any poll can be.

To learn more about what to expect in the 2018 House, Senate, and gubernatorial elections, download our Midterm Toolbox.