The 2016 Presidential Election was a shocker in many ways, and it has generated many bits of conventional wisdom which have been largely accepted in the mainstream culture. One of these is the “fact” that the polls had it wrong. After all, the polls all predicted a Hillary presidency, and, instead, we got the Donald. While this may qualify as what has been come to be known as an “alternative fact,” as far as regular plain old facts go, however, it just isn’t true – the polls were right, and rather exceptionally so given the closeness of the outcome. Here is some evidence.
Ten of Eleven of the Real Clear Politics (RCP) Average Polls Predicted the Correct Outcome: A fact that seems too easily lost in the discussion of the accuracy of polls in the 2016 election is that polls predict popular vote, not electoral vote. Ten of the eleven RCP Average polls in the last poll taken prior to the election predicted that Hillary Clinton would win the popular vote. As it happens, Hillary Clinton won the popular vote. So far, so good.
The Predicted Margin of Victory Was Very Close to Actual: Sure, the polls predicted the right (popular vote) winner, but in terms of the actual margin of victory, they could have been way off! However, they were not. The final vote count showed that Clinton defeated Trump by 2.09% – the average margin of Clinton’s win was 3.27%, or 1.18% higher than actual. The median and modal predicted difference was 4%, or 1.91% higher than actual. Not really all the much as it turns out. I have to admit it’s getting better.
Six of 22 Individual Candidate Predictions (RCP) Were Within Margin-of-Error: Now, 6 of 22 does not seem too good, but we need to flesh this out a little. Hillary Clinton’s predicted vote percentages were an average of .87% off from margin-of-error. President Trump’s percentages were off by an average of 1.58% from margin-of-error. The reason for this “poor showing” in terms of margin of error was the presence of 2 third party candidates in the race who received non-trivial percentages of the vote. Such candidates historically poll a little better than their final election day results – and this happened in 2016 as well. As a result, polls typically understated both Clinton’s and Trump’s actual vote percentages, in many cases taking the prediction slightly, but only slightly, outside of the margin-of-error. Yes, in terms of this criteria, the polls were wrong in 16 of 22 instances, but the extent to which they were wrong was trivial and provide no cause for the widespread flagellation (including self-flagellation) that polling companies have endured since the election.
Of course, the predicted winner was wrong, again, not because of the popular vote but the electoral vote. It could be that the polls led to faulty conclusions about the electoral vote, thereby lending credence to the “polls were wrong” mantra. Where we might find problems here, in particular, was in the exit polls. For a variety of reasons (not the least of which is the existence of widespread early voting) sampling in exit polls is typically less accurate than in other survey research methodologies. As such, predictions generated by these polls are likely to suffer when electoral outcomes are especially close (remember 2000?).
Three states in which the election results were particularly “shocking” were Michigan, Wisconsin and Pennsylvania. Despite relatively strong showings in pre-election state polls by the eventual president, Clinton was generally shown to be the leader, and this, combined with the electoral history of the states, led most observers (including, apparently, the Clinton campaign) to assume that the vote from these states would fall dutifully in line for the Democrat and Clinton would take her assumed place as the nations first female Chief Executive. This, of course, didn’t happen – Trump won all three states. Had he not, Secretary Clinton would have won, and the polls would have been vindicated.
So what of the exit polls from these three states? If we were to see evidence of real polling incompetence, we should find it there, especially given that the total Trump margin of victory for these states was a razor thin 77,744 votes, averaging .57% of the vote per state. Examining CNN’s published exit poll results, we find that, in fact, polls predicted a Clinton victory in each state. Nuff said, right? Not quite. As it happens, the margin of victory for Clinton predicted in each of these states was, in fact, within margin-of-error of the actual result. Despite the fact that the wrong winner was predicted, the polling results were right!
The problem, as it happens, lies not in our polls but in ourselves. In an effort to produce definite answers, we all too often forget that, ultimately, poll results are more accurately viewed as establishing an interval estimate (±MOE), not a point estimate. Taking factors such as margins-of-error into consideration, which we must, and considering the miniscule deviations from this margin discussed above, rather than hiding their heads in shame, pollsters should proudly proclaim the amazing accuracy of their results in the context of one of the most confounding presidential elections ever.
