Analytical Misstep? A Reality Check for Pollsters & Analysts

Image: Courtesy

On reviewing and reflecting on the recent social, political and economic events around the world and in my motherland Kenya, I can conclude with a high sense of surety and high degree of certainty that majority of the pollsters and analysts are getting their facts wrong. But when someone is getting his/her facts wrong on a certain issue, is it even worthy to classify such information as facts? Seems illogical.

Pollsters and analysts alike have been feeding the public with “statistical data”, analytics and analysis that have largely failed to mirror the eventual state of affairs. In the just concluded USA presidential elections, it is on record that the pollsters and analysts were calling for an out-and-out victory by Hillary Clinton in the Electoral College and most of the swing states. The finality; Donald Trump carried the day. In June this year, majority of the British citizens voted in favour of Brexit against the expectations of the pollsters and the analysts who had projected that the anti-Brexit camp would carry the day.

Back to the 2008/09 financial crisis and/or global recession, majority of the economists were unable to predict the economic meltdown. Even in Kenya, majority of the pollsters and the analysts usually make incorrect and ambiguous predictions about various phenomena. Revisiting the political situation in Britain in 2015 occasioned by the general elections at that time, the pollsters and analysts projected that there would be a “hung” Parliament with the Labour Party and the Conservative Party securing no outright majority.

Inherent & Systemic Weaknesses

There is no myth, magic or miracle that can be patched up and fabricated to try and offer explanations, clearly or amorphously, regarding the final results of these events in comparison with the incorrect projections. There are certain fundamental issues that are not being taken into account, and if they are considered then perhaps not with a lot of keenness.

To start with, the data collection habits of the pollsters and the analysts are poor. What do you expect to get when you carry out some shoddy work? In any case, Garbage In-Garbage Out. That is the starting point of all these skewed analyses that we are seeing. Since data analysis is a process, the quality of the input eventually determines the quality of the output. Due to poor data collection, the concerned entities/individuals end up formulating voter models/analytical models that are incongruent and inconsistent.

Another related and underlying weakness that is prevalent among the pollsters and the analysts is the failure to heed the principle of randomization. There is disregard for the random probability sampling technique and fashioning of the non-probability sampling approach. The random probability sampling is superior to the non-probability sampling method.

Random probability sampling involves the random selection of elements and in this case, every voter has an equal chance of being included in the sample. Hence, with the use of this system, the sample will be representative. On the other hand, non-probability sampling is based on personal judgement with the elements (Voters) in the sample determined selectively and not randomly. This results in a sample that is unrepresentative.

It is from the emanating unrepresentative sample (s) that cases such as the failure to interview the right mix of voters, getting opinions from potential voters who are easily contactable and under-representation of certain areas frequently occur. These events have subsequently led to the impossibility of figuring out the hidden/secret voters who were perhaps instrumental in actualizing a Trump presidency, Brexit and the Conservative Party garnering the majority in the British Parliament.

Furthermore, another structural weakness that manifests itself is the weighting bias. Most of the pollsters and analysts usually err at this point when they are seeking to make predictions of the various issues. They have the tendency of skewing the responses to match their projections especially if their funders are the elites who have the motive and objective of influencing public opinion. This weighting bias tends to occur when there are low response rates from certain cohorts of the likely voters.

Pertinent Questions

There are germane questions which ought to be put forth to try and figure out what the future has in store for the pollsters and the analysts.

a)      Is this the Armageddon for the use of science in predicting phenomena in social sciences particularly in politics? Not really. The pollsters and the analysts need to be more thorough in data collection and should exercise objectivity by employing the use of random sampling techniques though a bit expensive and time-consuming.

b)      What role do journalists have in the forecasting business? Journalists are after making headlines and therefore, they end up propagating horse-race journalism at the expense of giving incisive analysis on different issues. The greatest challenge lies herein; most media houses and journalists have the desire to carry out surveys about an issue in question but they hardly use the random sampling approach and they conduct their polls within two days or so. They end up with results that are biased and skewed to match their predictions. Most importantly, the media houses are owned by elites who are directly or indirectly involved in politics hence the need to uphold the doctrine of political correctness.

c)  Why are pollsters undergoing the irrelevance evolution? It’s pretty simple and open. They are corrupted by the political class to sway public opinion in their favour. This is certainly where the pendulum of subjectivity and irrelevance begins to swing.

d)     What is the missing link? The pollsters and analysts make a lot of assumptions when analyzing issues. They tend to assume that the voters are rational just like the way economists assume that consumers are rational. It is on the basis of hinging on rationality in voting behaviour and patterns that we end up with forecasts that are not correct. The principle of rationality is a detachment from reality and it is too academic.

After making consistently incorrect forecasts, the pollsters and analysts affected should go to the basics and rectify the situation. They should scale down on bottlenecks such as the use of non-probability sampling methodology and being corrupted by the political class. Solving the former is easier, but the latter is quite difficult because of the intentions, manipulations and machinations of politicians to influence public opinion. The worst of it all lies within the doorsteps of the media houses which have the pressure to be politically correct. Otherwise, we can keep on thanking the misleading pollsters, analysts and media houses.