Interpreting polls and election data - guidance for media and journalists

People still find value in polling which is why it continues today.  It is part of the expression of free speech – and many people like to express their views by participating in polls – and it provides evidence to guide better policy decisions.  Polls contribute to an understanding of people’s attitudes and beliefs which is important to a huge range of organisations and businesses. 

• Wider market context if asked how polling methodologies have changed

Many polling companies have carried out reviews of their methodologies over the past decade. Some have invested in techniques such as assessing emotional resonance, to determine the intensity of people’s feelings about a question or topic. Others have focused more on in-depth research about public views and attitudes to complement or substitute for more traditional voting-intention polls.

Multilevel Regression and Post-stratification (MRP) polls are increasingly being conducted to predict outcomes in individual constituencies based on national surveys.  They achieve this by combining national survey data with local census figures, such as the age and income distributions of voters in that area, to give a more accurate prediction of election outcomes.  As these polls require more data and analysis, they are typically more costly.

In short, they don’t necessarily come out in favour of the party or paper who commissioned them.  Much of this perception comes down to how the data is presented and the facts which journalists choose to use in their writing.  

If you are interested in reading a poll’s results in full and the pollster is a British Polling Society (BPC) member, this information should be available via the BPC website as all members commit to making their findings available to the public.

It is not for us to advise on this. Whether our members and their clients choose to commission research around a forthcoming General Election, is a commercial decision for them to make. 

From an MRS perspective, we have twelve key principles and sixty-four rules – our MRS Code of Conduct – which we expect our members to abide by whilst undertaking their professional activities including any polling. 

The Representation of the People Act 1983 prohibits the publication of the results of polls conducted on Election Day whilst voting is taking place

This is outlined in Ofcom’s (the communication services regulator) Broadcasting Code.

Generally, polls are a snapshot of people’s opinions at a given point in time, and these views can change between when the poll is carried out and when ballots are cast.

Multilevel Regression and Post-stratification (MRP) polls combine national polling results with local census data to predict outcomes in individual constituencies.

Exit polls, however, are completed post the ballot.  They therefore measure how people have voted, rather than asking the electorate their intentions if they were to decide to vote. 

Regardless of the type of poll, they should always be transparent. The methodology, and who commissioned the poll, should always be published as the context of a poll is often just as important as the findings.

The MRS Code of Conduct sets out the guidelines for how reputable research should be carried out. If you are in any doubt as to whether a poll has been conducted ethically then I urge you to contact MRS Codeline [codeline@mrs.org.uk].

If a polling company is an accredited member or Company Partner of MRS there are strict standards which must be followed and if you feel that this has not been the case, a complaint can be made and the organisation in questions will be investigated.  

Pollsters get it right most of the time. Market and social research in all its many forms plays an important role in our society.  From charities to the government, media and business, it is a tool which supports strategic decision-making by helping organisations understand their customers, clients and end-users.

There are numerous examples of how research has helped improve people’s lives.  From informing health and safety campaigns such as giving up smoking to helping work out how products can be reformulated to cut sugar and salt.  Media organisations and media regulators such as Ofcom also rely heavily on research to understand changing patterns of media consumption.

There have been instances where polls have incorrectly predicted results, due to unrepresentative samples, a failure to account for margins of error or late ‘swings’ in voting intentions.  If a polling company is an accredited member or Company Partner of MRS there are strict standards, set out in our MRS Code of Conduct.

Members must abide by the MRS Code of Conduct and follow recommendations of best practice. However, polling will only ever be a snapshot; a temperature-check at a specific point in time. 

Statistically, a sample of 1,000 adults across the UK is sufficient to provide a good estimate of opinion.  Reporting on a specific sub-sample group of, for example, 100 participants would need more care to ensure the results are statistically significant – that is, that they are not likely to change noticeably depending on the participants selected.

A certain margin of error should always be factored into reporting on and understanding of poll results.  For national polls, this is five per cent.  A swing in opinion or voting intentions of three per cent, for example, would not be considered statistically significant and therefore should not be used to support news reporting.

For smaller sub-samples, this margin will be greater, to account for the potential impact which the selection of participants could have on the results.

Opinion polling accounts for around 1 per cent of the research sector as a whole.  Political polling accounts for a small proportion of that 1 per cent.

It is impossible to say as many organisations may call themselves polling companies but do not have the credentials of a reputable provider. 

The only way of knowing whether a research organisation or practitioner is reputable is by ensuring that they are accredited by the MRS and as such can be held to account on best practice. 

This level of due diligence is particularly important due to the marked increase in the number of uncertified – and sometimes politically motivated – pollsters publishing findings in recent years.  This trend is contributing to the rise in disinformation seen within the media and online, making it ever more important to check the sources of any data.  The full list of MRS-accredited pollsters can be found on the MRS directory and the BPC website.

MRS has over 700 company partners and just under 5,000 individual members. It is not possible to break out exactly which companies and individuals provide polling as this is not a static figure.

The issues surrounding polling predominantly concern future voting intention and are specific to pre-election polling. 

Market, social and opinion research is a UK success story, contributing £9bn to the UK economy.  We are a world leader in market and social research precisely because of the quality and qualifications of the talent in the sector and its ability to learn, adapt and innovate.

AI is an extension of machine learning, which the market and social research has been using effectively for some time.  Generative AI has the potential to enhance the work of pollsters, helping them to gather and analyse findings more efficiently.

However, there are also certain challenges researchers are having to manage as a result of AI.  First, machine algorithms are only as good as the data which is put into them and this can often be biased due to the way they are programmed, and can therefore favour the majority and exclude the voices of minority groups. 

AI tools have been known to ‘hallucinate’ and fabricate data, so everything they produce must be thoroughly checked.  This is also the case for verifying data to ensure that responses are from real people, and not from bots.

MRS has developed The BEST Framework for Gen AI, advising the research sector on how best to build AI into research processes, and has issued MRS Guidance on Using AI and Related Technologies setting out the ethical principles for the application and use of AI in research.

Polling will only ever be a snapshot; a temperature-check at a specific point in time. We await to see the results of the next General Election but if needed we will convene to assess if any changes to our Code of Conduct or guidance to members is required. 

We have also been working with journalists and the wider media, such as through our guidance developed with media regulator Impress on how to interpret and use polls within journalism, to help ensure that polls are responsibility reported. This includes explaining what the margin of error is and communicating this to consumers so that they are aware when polling results fall within this range; likewise making clear if a poll is demographically representative – or not. 

For more information see the MRS website.

Market and social research, and polling as a subset of it, is successfully self-regulated. This is what the findings of the Lords Inquiry into Polling found when it investigated the matter in 2017.

The recommendations made by the House of Lords Inquiry, chaired by Lord Lipsey, were that MRS and the BPC should work closely together to promote transparency and help to educate journalists on how to accurately report on the polls, which we have done.

MRS has a Code of Conduct – a set of professional and ethical standards that all research practitioners must maintain. 

This Code is updated regularly to ensure it reflects evolving practice.  The most recent changes to the MRS Code of Conduct took effect in August 2023. Among the changes was a tightening of reporting obligations to ensure samples are representative of segments of the population.

We also work with press regulator Impress to promote best practice among journalists on how to interpret and report on polls and surveys.  This includes explaining what the margin of error is and communicating this to consumers so that they are aware when polling results fall within this range; likewise making clear if a poll is demographically representative – or not.