How to access the International Journal of Market Research (IJMR)
Published by SAGE, MRS Certified Members can access the journal on the SAGE website via this link.
Margins of error - statistics in the media
19 June 2013
Share
This was the title for a packed evening seminar held at Kings College London on 14 May, jointly run by RSS/King’s College/Ipsos MORI.
Subtitled ‘Public understanding of statistics in an era of big data’, this is one of a series of events celebrating the international year of statistics.
The aim of the evening was to explore why statistical literacy and trust in statistics remains relatively low, identify the implications and recommend solutions to improve understanding and trust. A great evening, with excellent and eminent speakers, but far too little time was allowed for discussion of the important issues they raised.
I wanted to ask if one problem was the devaluing of statistics in the media, the ‘a survey says’, or as Joan Bakewell eloquently said in a Guardian article some years ago:
'...surveys themselves need checking out. They are not value-neutral, laboratory-style findings.
'They are often done in the street – in bad weather – by people with clipboards, who would prefer to be doing something else, with passers-by who do their best to avoid eye contact but relent before the supplicant’s ‘It will only take a minute’. So why is this far from exact method used as reliable evidence for claims that can frighten us, threaten us and cajole us – but rarely reassure us?'
This leads to the challenge for the public in detecting ‘the signals in the noise’ – distinguishing the serious statistical messages from the trivia, or worse…
I didn’t get the chance to ask the question, but if I’d read that day’s Guardian more thoroughly before I attended, I certainly would have made sure I did. After getting home I found an article headed ‘Gove’s attack on pupils’ grasp of history relied on PR surveys’.
A Freedom of Information Act request had identified that our current Education Secretary’s attack on history teaching was based on press releases of polls conducted by UKTV Gold and Premier Inns, Lord Ashcroft and the Sea Cadets!
Gove is quoted as saying in the Mail on Sunday "Survey after survey has revealed disturbing historical ignorance, with one teenager in five believing Winston Churchill was a fictional character while 58% think Sherlock Holmes was real".
As the Guardian article claims: 'None of the pieces included links to the original research, and none of the articles cited stated whether the research was commissioned by professional polling companies, or met the standards of the British Polling Council’ – or I would think, met best market research practice and the CIPR/MRS/RSS guidelines? Who knows!
At the seminar there was much talk of the need to build trust; apply codes of practice etc, but if government ministers can make controversial statements which have potential major policy implications based on such shaky statistical foundations, what hope is there for us to build trust in statistics with the wider public, or with key professional groups in society such as teachers?
As Bobby Duffy demonstrated from Ipsos MORI data, 54% of UK adults have no trust at all in politicians and the proportion has steadily fallen over the years, and for journalists and government ministers.
It is also not surprising, as pointed by Denise Lievesley, that trust in statistics within the UK is also very low, compared to other EU countries. We have a mountain to climb!
More ijmr newsletters
Get the latest MRS news
Our newsletters cover the latest MRS events, policy updates and research news.
©2024 The Market Research Society,
15 Northburgh Street, London EC1V 0JR
Tel: +44 (0)20 7490 4911 Email: info@mrs.org.uk
The Market Research Society (MRS) is the world's leading research association. For all those who need, generate or interpret the evidence for making good business and policy decisions. Click here to find out more about us.
©2024 The Market Research Society,
15 Northburgh Street, London EC1V 0JR
Tel: +44 (0)20 7490 4911 Email: info@mrs.org.uk
6 comments
Hilary Burrage20 Jun 2013
You will be interested to know that The Economist has also addressed this issue recently: http://www.economist.com/news/britain/21579016-coalition-government-showing-worrying-disregard-data-stats-spats-and-spads . In their words, "... cabinet ministers operate like feudal barons, rarely sharing research or working with colleagues.... Academics who rely on census data, such as Danny Dorling, a demographer at the University of Sheffield, fear that the government is determined to scrap the census no matter what." Why isn't this front page news? We have been warned....
Peter20 Jun 2013
Thanks, Hilary, I completely agree with your view about the Census, a topic I covered in an earlier blog. I also return to the abuse of stats in my next Editorial in IJMR (55/4 in July).
Hilary Burrage20 Jun 2013
Cheers Peter. Of course the Census is not the 'only' (!) matter of concern. It's also terrifying that apparently Ministers are knowingly mis-using and otherwise abusing statistics to over-ride what would with proper data take them to different conclusions and outcomes. The Economist doesn't usually overstate the case in matters of this sort! Who should those worried about all this turn to? Hilary
Mike Thompson02 Jul 2013
Before we educate everyone else about stats I think we should start at home. The level of knowledge amongst MR professionals is lamentable. However, if we encourage or at least do not tell the truth about access panels on how unrepresentative they can be we are not going to get very far with making our profession more awrae of even the basics of stats. It is apparent that most access panellists appears to be carried out with people working with around 5 access panels and completing around 6 surveys a week (this comes from a meeting that was held at an MRS meeting concerning access panels) a year or so ago. I would be interested to hear from access companies concerning the implications of this and also from clients that know this but still use them. I should admit we have to use such panels ourselves because the client asks for them although I make sure they understand the possible implications.
Annie Pettit03 Jul 2013
It disappoints me when people point out the inherent problems of access panels. When it comes to obtaining sample for market research purposes, there are NO probability samples anywhere. Focus groups, in-person interviews, communities, access panels, social media research - ALL of these suffer from non-random sampling. Heck, even though it is legally required to answer the census, the response rate is never 100%. People will never comprise a random sample. So the issue is this. Get over it. Understand your sample. Know the strengths and weaknesses. Learn how to generalize appropriately. If you're unsure, say you're unsure. Market researchers aren't gods. Rant over. :)
Dick Stroud13 Jul 2013
Peter, thanks for this blog posting. Interesting and a little depressing. We seem to be living in a ‘research says’ culture where politicians, the media (especially the BBC), companies and NGOs find it necessary to sprinkle a few ‘facts’, often from unpublished research, into their messages to gain greater impact. Worse still, I suspect that the audience filters these research-based messages to reinforce their own prejudices. I am not sure how we improve the situation other than being sensitive to the problem. This then raises another issue – do we only challenge those ‘research says’ facts that we disagree with?