1989 landmark paper warns of fresh dangers in reporting of opinion polls
07 August 2015

'Opinion bandwagons in attitudes towards the Common Market'

Catherine Marsh and John O’Brien, JMRS Vol 31 No 3, July 1989

The latest quarterly Landmark Paper is slightly late as we have to specially digitise selections from earlier issues of the Journal. However, the decision to not limit the Landmark Paper to papers already digitised enables me to choose papers from the entire run of the MRS Journal, stretching back to 1959.

With the prospect of a referendum on UK membership of the EU now built into the current government’s programme, I thought it might be interesting to see if the EU/Common Market membership as a topic had featured in any paper published in the Journal. It can’t get any better than this selected paper, authored by two of the leading researchers of their generation. The late Catherine Marsh was at the time the Visiting Simon Fellow at the University of Manchester, on leave from her role as a lecturer in sociology at Cambridge University. John at the time was Deputy Chairman of BMRB, and Chairman of the UK market research sector ‘think-tank’, the Market Research Development Fund.

As you will see, the authors describe an experiment conducted to see if different opinions given to random-split groups provide different results. As the authors point out, few experiments in this field are designed to measure participants’ perception of the trend, rather than attempting to alter perceptions of public opinion. The research design for the experiment, using the BRMB telephone omnibus, is fully described, including the four questions used to measure opinions, including one on how participants think opinions on the Common Market are moving (Britain staying in; Britain exiting; opinions not changing). As such this question mirrors the wisdom of crowds approach developed by ICM (see ‘Predicting elections: a ‘Wisdom of Crowds’ approach’, Martin Boon, IJMR 54/4, 2012). The overall approach built on an earlier survey on opinions towards abortion. The test treatment was introduced after that question for reasons described in the paper, with the wording: ‘recent opinion polls have shown that public opinion is moving towards Britain staying in/getting out of the Common Market’. The findings, with participants stated view if they were to vote in a referendum as the dependent variable, show a marked effect on attitudes. The authors show how the effect changed across sample sub-groups, particularly whether they were pro or anti Common Market membership.

In their discussion of the findings, the authors show that the topic can affect attitudes, as shown in the comparison with the earlier survey on opinions of abortion using similar methods. They discuss the impact of a ‘bandwagon effect’ and the difficulty in developing a conceptual basis from the diverse factors identified in research into this effect. From their findings, the authors identified three groups: the ‘ignorant’, most affected by the trends in opinion; ‘neutrals’, pro-anti Common Market and most strongly affected; ‘dispassionates’, with no strong feelings and least affected. They discuss what leads to attitude tenacity, and who is likely to change their views in the direction of a bandwagon.

They conclude the paper with the following warning, one that seems highly relevant in the current climate of distrust in opinion polling following on from the referendum in Scotland, the recent general election and the polls likely in the run up to the EU referendum:

‘So, if changes in public opinion are not only more likely to be reported and more error-prone but also more influential than static ones, there could be a powerful and potentially dangerous cocktail at work in much reporting of opinion polls. It merits further investigation’. 

The irony is that on the one hand it was the single poll in Scotland suggesting a change in opinion to favour leaving the Union that ignited the campaign last year, whilst on the other hand, it was the static picture in the UK in the run-up to the general election that led to the firm belief in a coalition outcome. It’s not just what the polls report, but also how they are reported in the media that matter in influencing the final outcome on the day. In the light of the recent performance of the polls, at least until the BPC/MRS enquiry is completed, I’m sure that the media will be wary of how they report the polls in the run-up to the referendum, but the evidence is there that reporting of static results, as well as changes in opinion, can both misrepresent the overall mood in the country, thereby misleading voters and politicians, and therefore need to be treated with caution.

Read the full paper here