Landmark Paper: How do you like your data: raw, al dente or stewed?
30 July 2014

The latest Landmark Paper is drawn from the two special issues of JMRS, celebrating the 50th anniversary of the MRS. It was originally published in the Proceedings of MRS Conference 1985, and presented at that event by the authors.

The theme of the paper addresses what should surely be a fundamental concern to all readers of IJMR: ‘how well market research data mirror the realities which they are intended to measure’. 

You can view the paper here:

How do you like your data: raw, al dente or stewed? Tim Bowles (IRI InfoScan), Bill Blyth (Taylor Nelson AGB) JMRS Vol. 39 No.1 1997

The authors also discuss the options open to researchers when the data apparently fails to meet that objective. As the authors point out, tests of external validity can be difficult to apply to many categories of research, but in areas where market research sourced data is the key source for decision making, such as when allocating multi-million pounds of advertising spend, clients rightly demand re-assurance that the results are accurate. 

Market measurement, through continuous data collection methods, especially panels, often underpins such decisions, and wider ones on brand and product strategy.

The authors provide a fascinating overview of market measurement in the 1980s, when EPOS was still being rolled out in retailing, and before the advent of loyalty cards. The tools to collect data on consumer purchasing may have changed, but the issues of accuracy and validity remain vitally important today in terms of measuring the whole market – consumers are seldom totally loyal to one retail chain, or in the brands they consume. 

Therefore, market research methods remain a fundamental source of data for market measurement. Credibility of the output is also as important as ever.

The authors consider ‘bias’ in some depth. In addition to listing, and discussing in detail, the areas where potential effects might apply in continuous research, the authors discuss what the then content of the MRS and ESOMAR Codes of Conduct had to say about identifying potential sources of bias in research design. 

Whilst the term ‘validity’ remains within the 2014 MRS Code, mentions of identifying possible sources of ‘bias’ have long since disappeared.

The authors claim that, even then: ‘Technique limitation is rarely recognised, at least in public. This should not be so…Research does have its limitations. Better that they are identified and worked on, and at the same time subjectively and objectively allowed for, rather than putting our heads in the sand, which leads to……’. (That’s how the authors finish that sentence). 

Hear, hear – but is the situation infinitely worse today? Where would such discussions take place? 

The research sector has changed immeasurably since then, in terms of ownership and commercial confidentiality. In those days, in the UK, the Research Development Fund (later the MRDF) existed as a forum for exploring and discussing methodological issues, but that is also long gone. 

The recent National Centre for Research Methods (NCRM) symposium on web surveys (see Editorial, IJMR Vol.56 No.4) was one opportunity, but driven by the social, rather than market, research sector.

The authors’ opening sentences also demonstrate that the market research sector was even by 1985 starting to face a new world, but not everyone shared that vision: “Among a certain group of market researchers it is now taken as self-evident that market research is part of the wider ‘information business’. In the past, however, market research has been primarily concerned with data, not information”. 

They also saw the conflict that might develop if researchers had to drop their ‘scientific pose’ in order to play a wider role in data manipulation and prevent the rise of specialist data processing companies, with market researcher relegated to simply a data source. 

Sounds familiar? By the time the paper was re-published in JMRS in 1997, the authors were even more sure of their ground and the changes that market researchers faced, stating in their new forward: ‘A subsidiary message of our paper was that more and more market information would become available as a by-product of electric data systems in retailing and distribution. This would require the market researcher of the future to make a shift from being mainly a data collector, to being a broker and analyser of data which belongs to others’. 

Seventeen years on, the debate still rages on where market research(ers) fits into this wider world, fuelled in recent years by the impact of ‘big data’.

So, whilst at one level this paper provides an interesting historic perspective of market research methods for measuring markets of nearly 30 years ago (see the methods for TCPI Market Track, AGB Lek Trak; TGI), key issues raised by the authors remain as valid today as they did then. 

However, firstly, I wonder if such a paper might be written today; and secondly if it was, where would the opportunity be to discuss the content?