Predictions of election based on social media data completely miss the mark
Some of those claiming the polls dramatically failed at this election assert that we should rather rely on data from social media, which are allegedly less error prone. We are talking buzzwords such as big data, machine learning and artificial intelligence, so what could potentially go wrong?
A great deal, it turns out.
(Disclaimer: I found the predictions through quick and dirty Google searches, which would reveal the most visible ones, not a systematic review. There may be some more out there that fared better than the ones I found.)
Among the 10 different predictions based on data from social media I did find, 8 of 10 were completely and utterly wrong, thus underperforming 10 random chimpanzees by a wide margin. However, two of the ten were quite accurate in their predictions.
Most of them analyze Twitter. It is fascinating to note how the same data can result in diametrically opposite conclusions (although the general tendency is skewed in favor of Trump). It is also interesting that few of these approaches problematize 1) the degree to which the Twitter universe reflects the real world and 2) whether sentiments expressed in social media translate into votes. Instead, they generally focus on how BIG the data are, as if that solves all problems of systematic bias.
In spite of technical sophistication, beautiful graphs and high number of observations and decimals, analysis of social media data for election forecasting is an approach still in its infancy. There are no standardized tools or methods with high reliability and validity. In the longer term, it may be a useful supplement to polls, but I think we have to wait a while before leaving election forecast to analysis of data from social media.
Here are the predictions I found:
1) Two Swiss models predict a Trump win!
Two teams of researchers from Switzerland examined search frequency and “speech analysis” and concluded that a decisive win for Trump was likely. Their argument was that the bigger data you have, the better: “When you have big, big data sets like we have, from all the [relevant] channels, then you can identify trends and that makes it easier to make predictions”.
https://www.swissinfo.ch/eng/why-two-swiss-led-data-models-predict-a-trump-win/46131826
Verdict: Complete miss.
2) Trump will win US election with 270–280 seats!
The most high profiled and widely reporterd prognosis came from Professor Bela Stantic, who collected 4 million tweets with 20 million likes and analyzed them, “ and based on that analysis I concluded Trump is ahead and he will win with 270–280 seats.”
https://www.skynews.com.au/details/_6203730374001
Verdict: Complete miss.
3) Trump to win elections at 93% probability!
The company ICONIC were almost certain Trump would be reelected, based on Google searches. According to them, such searches have none of the weaknesses of surveys, such as non-response, social desirability bias etc.
Verdict: Complete miss.
4) It’s likely that Donald Trump will stay the American president for another four years!
Awario used a “social listening tool” to analyze social media content (March to June this year), with a focus on mentions, sentiment, demographic data and different themes. The conclusion was that Biden was trailing, but he still had the opportunity to turn the trend — if not, Trump would win. In reality, Biden was ahead at that time, with Trump catching up somewhat towards the end of the campaign.
https://awario.com/blog/who-will-win-elections-2020/
Verdict: Complete miss.
5) Twitter analysis shows Trump on course to win US election!
Researchers at Loughborough University trained a «unique» computer program that could analyze 3000 tweets per second and examined Twitter activity mentioning Trump or Biden. The conclusion was a clear win for Trump.
https://www.lboro.ac.uk/news-events/news/2020/november/twitter-analysis-shows-trump-set-to-win/
Verdict: Complete miss.
6) Research paper: Trump wins (with a seven decimal precision on the estimate)!
Three researchers co-wrote the paper “Prediction of USA November 2020 Election Results Using Multifactor Twitter Data Analysis Method”. Unlike previous research, they combined different characteristics of tweets. Apparently, that didn’t help much as they concluded: “Based on these scores, it is concluded that the Republicans will win the election with a very small margin.” The result was presented with seven decimals.
Verdict: Nice graphs, but miss.
7) Biden wins with 350 electoral votes!
The PEORIA project at the George Washington University Graduate School of Political Management (GSPM) also analyzed Twitter data, but in combination with polls. This analysis ended up with skewing the prediction further in Biden’s direction — 350 electoral votes and wins in Iowa, Texas og Ohio.
Verdict: Miss.
8) Biden wins with 501 electoral votes!
It is perhaps unfair to include this one, but this blogger used Twitter data to predict the election and ended up with a whopping 501 electoral votes for Biden and only 37 for Trump: «The results I got where quite odd but given the current events and the fact that we are in 2020, everything is possible!”
Verdict: Complete miss.
9) Biden wins popular vote by 50–47.
There are exceptions: The Italian company expert.ai used sentiment analysis of Twitter data and ended up with an astonishingly correct prediction: “The company’s analysis puts Democratic candidate Joseph Biden ahead of President Donald Trump, 50.2% to 47.3%, a margin that is much narrower than the double-digit lead that Biden has over Trump in most national opinion polls.”
https://fortune.com/2020/10/14/polls-trump-biden-presidential-race-artificial-intelligence/
Verdict: Bull’s eye.
10) Biden clearly ahead.
Finally, Polly Pollster, the artificial intelligence pollster that correctly predicted the Liberal’s minority win in the 2019 Canadian election, estimated Trump’s chances of winning the White House as slim — i.e. at about the same level as fivethirtyeight.com.
Verdict: Bull’s eye.
