Global data quality

MRS is coordinating efforts with the Association for Qualitative Research (AQR), The Canadian Research Insights Council (CRIC), ESOMAR, Insights Association, the QRCA, The Research Society (TRS), SampleCon, and The Association of Market Research Austria (VMÖ), and the Global Research Business Network (GRBN) to address ongoing and emerging risks to data quality in the market and social research, consumer insights and analytics industry. With the goal of increasing information and building trust, each organization will lead a workstream that delivers to the global quality resources to improve the conversation and outcomes around:

• The Language of Quality - how we refer to the different aspects of fraud, duplicates, and survey cleaning in ways that inform with accuracy and transparency
• Fraud detection – tracking the prevalence of fraudulent survey completions by humans or bots and outlining best fraud detection and mitigation practices
• Identification and mitigation of bias from sample frame and representativeness
• Data quality in research surveys, and the resulting impact on overall quality of the data
• Improvement in the research participant experience

You can find links to all the organisations and their work via the Global Data Quality website.

MRS Data Quality Guidance

Approaches being used across the sector to combat data fraud including a review of some of the legal considerations for each of the approaches detailed.

Glossary of terms

With a lot of technical terms to consider, the goal of the GDQ glossary is to refine the way we talk about data quality and become more precise in describing the challenges.

Go to the glossary

Why is this initiative needed?

These issues are global and persistent and must be addressed in a concerted industry-wide manner. We can more effectively and efficiently combat data quality threats by coordinating our efforts.

The threat of fraud in research isn’t new. However, fraudulent activity is becoming increasingly sophisticated, particularly in online research. It poses a significant risk to our sector’s future and this international partnership is an important means to confront it. 

Data is at the heart of what we do. It’s important we remember that not all data is created equal. Poor research can have a catastrophic impact on decision making and, as businesses and organizations across the world face economic, political, and social challenges, now more than ever our sector needs to prioritize the delivery of quality data and insight.

This partnership helps foster a collaborative, honest, and transparent discussion of data quality issues, which is essential to the evolution and wellbeing of our sector. We must fortify trust in data insights.

This is a truly collaborative industry effort, working with skilled research professionals to provide the insight and tools to support research users and sustain trust in the collective knowledge that our industry generates.

Current and planned work

Each organisation is leading an effort to create content, standards, and training around key functional areas of quality including language, measurement, activation, participant experience, and sampling technology.

The organisations are committed to facilitating forums to raise awareness and encourage presentations by and discussions with experts and thought leaders. Meetings are already underway in which experts are working collectively to formulate best practice guidelines and buildout, refine, and coordinate universal terms and definitions. Research, including fielding a survey about fraud detection, is being discussed.

MRS is focused on fraud and bot technology

The MRS workstream comprises three main topics:

• Fraud and bots technology
• Mobile optimisation considerations
• panel/supplier data analytics

Fraud and bot technology is a complex and fast moving area. MRS is addressing the issue with a multifaceted approach addressing the current causes, identifying tools and methods to address the issue and developing new tools to enable the sector to keep pace with technological developments including the challenges posed by generative AI.

The project is structured around eight projects:

• Creating a glossary of the terms and definitions used to describe bot and fraud technologies
• Identifying approaches being used by the research sector to combat bot and fraud technology across modes and methodologies and providing guidance on the costs, resource and time implications of each approach
• Identifying approaches used by other sectors to address bot and fraud technology and determining which if any might be appropriately used by the research sector
• Identifying potential legal and GDPR challenges arising from the techniques used to address bot and fraud technologies and providing guidance on how to address these issues
• Compiling a list of sources of fraudulent responses and working with the platforms to have the sources removed
• Creating new solutions to combat bot and fraud technologies with the formation of a collaborative tech lab
• Investigating how technology is disrupting qualitative research and providing guidance to mitigate the risks of fraudulent participants
• Documenting and categorising third party quality and security software solutions

A multi-disciplinary working group has been created by MRS to address data fraud and bot technology:

• Tom Bates - Research Bods
• Joanna Bygrave - NIQ/BASES 
• Cecile Carre - Ipsos
• Oscar Carlsson - Milo Advisory
• Rebecca Cole - Cobalt Sky
  
• Shifra Cook - Ayda
• Daniel Culshaw - B2B International
• Liz Diez - Acumen
• Shelby Downes- Cint
• Mardien Drew - Today Consultancy
  
• Sandy Franklin, davies+mckerr and the AQR
• Timothy Frewin, System 1 Group
• Simon Glanville - Ronin
  
• Tracy Isacco - L&E Research
• Marina Joshi, Kantar
• Rita Kite - GfK
• Florian Kögl - Redem and the Austrian association,  VMÖ
• Joanna Price, Kantar
• Nicola McCully, Kantar
• Kerry McKinnon - Dentsu
• Leah McTiernan - independent consultant
• Jack Millership - Zappi
  
• Vitalija Narstyte - MM-Eye
• Tabita Razaila - Research Bods
• Jospeh Schiappa - GfK
  
• Chetna Sharma - Kantar Profiles
• Aaron Simmons - Dynata
• Jerome Sopocko, Askia
• Chris Stevens - consultant
  

Mobile optimisation

For over five years MRS has been conducting research into mobile optimisation exploring the impact that poor mobile design and lack of mobile optimisation has  on participation, completion and response rates. Dynata, Kantar, Cint (Lucid) and Toluna – have been working with MRS giving access to their response rate data since 2016 to identify trends in participation and response rates.

Drawing from the research and the experiences of the four companies, MRS has produced some best practice recommendations to help practitioners to produce better mobile design, increase optimisation and to improve completion and response rates.

This guidance has been updated in 2023 incorporating some of the key findings from the 2022 wave of the research. This guidance is supported by the US Insights Association and The Research Society of Australia.

The topline research results from 2022, the webinars presenting the results for each wave of the research plus the MRS best practice recommendations are available here.

MRS is also investigating whether a standardised set of panel/supplier data analytics can be developed to track and monitor participant satisfaction.

You can find links to all the organisations and their work via the Global Data Quality website.

Resources