US election: how did the polls get it so wrong?

Well, folks, this should be the end of industrial political polling as we know it.

Republican presidential nominee, Donald Trump began his tilt at the presidency as an outlier, and even as recently as voting day was characterised by many pollsters as the likely loser.

Now he is the US president elect, an outcome that most mainstream observers regard as an astonishing upset. Comparisons with the Brexit surprise abound.

The result underscores how badly modern polls serve us. Polling is an enormous, lucrative and influential industry – Australia has lost sitting prime ministers over poll results. But the US election outcome shows how unreliable polls can be at predicting our political futures.

Humble pie

Many pollsters are now eating humble pie.

Sam Wang, the widely-lauded pollster behind the Princeton Election Consortium website, was only recently so confident of a Clinton win that he tweeted:

As the votes for Trump rolled in, Wang wrote that he’s getting ready for “bug-cookery”, saying that “the polls were off, massively”.

In the late hours of voting day in the US, Wang wrote:

The entire polling industry – public, campaign-associated, aggregators – ended up with data that missed tonight’s results by a very large margin. There is now the question of understanding how a mature industry could have gone so wrong. And of course, most of all, there is the shock of a likely Trump presidency. I apologize that I underestimated the possibility of such an event.

Nate Silver, who uses statistical analysis to crunch poll data on the website FiveThirtyEight and famously called the outcome of the 2012 election correctly, was less confident than Wang of a Clinton presidency. But FiveThirtyEight did tip Clinton as the likely winner:

Nate Silver seemed to grow reflective as the results became obvious on the evening of voting day:

In the space of a few hours, GOP pollster Frank Luntz went from predicting a Clinton win to declaring Trump the likely next president:

He wrote on Twitter that exit polling margins “were way off”.

Polling in the past

Polling used to be a lot easier.

Polling emerged for sound economic and political reasons. On the economic side, advertisers needed a trusted way of reading the customer mind, so they could design products and develop media plans to sell them. On the political side, legislators, political candidates and planners needed reliable means of reading the citizen mind, not only for elections but also for everyday policy affairs from welfare to housing provision.

Not surprisingly, early polling methodology experts understood that devious people or organisations might attempt to distort the outcomes of ratings and polls. In advertising, companies that subscribed to ratings and actively tried to distort them would be threatened with removal from the subscription and banned from accessing ratings.

But polling technology has changed, and perhaps so too has community willingness to participate in them.

A cultural and technological shift

Thirty years ago, pollsters could use landline phone numbers to amass a statistically representative group of people. For example, reverse telephone directories ensured that pollsters would be fairly confident on their sampling frame and the identity of participants.

The shift to mobile phones and internet polls have meant that pollsters are now faced with a more fluid situation. Not as many people are willing and able to take calls from pollsters on their home phone – if they even have one – which can affect the quality of the data. And after all, pollsters can only work with the data they have.

It might appear that with the rise of the internet, there are now easier and more reliable ways to divine the citizen mind. But what is gained from “rich data” (what people say in detail) has to be balanced against the loss of a reliable sampling frame – not everyone has access to the internet or the time to fill in internet polls. More importantly, perhaps, unlike the earlier conventions polling is no longer deliberative and participatory.

Who cares about polls?

Technologies for reading the public mind, especially recently, appear more important to the media than they are to citizens. People do not vote based on the polls.

There are usually several, slightly different elections forecasts and poll predictions are quite obviously perceived selectively in favour of one’s own opinion.

New methods of reading the public, citizen, mind are needed. Contemporary technologies of political polling are no longer up to the task.