The 'decomposition' of market research
07 July 2014

It’s a lovely summer’s day where I live, as I write this blog. The countryside is quintessential English, with the smell of new mown grass in the air, the trees and hedgerows still fresh green, with wild roses and honeysuckle in flower. 

But despite the glorious weather, I feel a tad gloomy where market research is concerned, so I’m sorry but this will not be an upbeat read.

There are three reasons for this feeling:

1. Where’s science gone?

The first is my worry that we are leaving behind the sound scientific based principles that used to underpin what we called ‘market research’. At a recent symposium organised by the National Centre for Research Methods, ‘Web Surveys’ of the General Population’, it was once again underlined that commercial online access panels deliver variable results and do not facilitate replicability. 

The depressing finding was that despite the wealth of experience gained over the years since these panels were first introduced, the quality issues remain unresolved. Quality is traded off against cost and speed, but where is value for money in client thinking? 

Without a sound basis for sampling, where is reliability in the findings? For much more on this important symposium, see my Editorial in IJMR 56/4 (July issue). 

Engaging in other new scientific developments, such as neuroscience, has its limitations if the foundations for applying these techniques are suspect. I shall be returning to this theme later on in the summer.

2. Are algorithms the future?

Turning again to issue 56/4, you will find two papers covering different perspectives on using software for text mining. As social media becomes an increasingly important source of ‘insight’, the need for ever more sophisticated software based analytical tools to search for patterns in the data, and forecast likely future behaviour, in this sector rapidly grows. 

As one of these papers describes, it is now possible to train the software to trim the costs of using human coders who assess the accuracy of what the software is delivering and advise on calibrating the performance. 

As the market research sector increasingly embraces social media and other sources of passive data, the use of these tools becomes a norm.

3. The decomposition of market research

Richard Susskind, who has already described the impact of sophisticated IT solutions on the legal profession (‘Tomorrow’s lawyers’, Oxford) has, I gather, broadened his perspective looking at the likely impact on all professions in a book shortly to be published ‘Beyond the professions’, co-authored with his son Daniel Susskind. 

In this they introduce the concept of ‘decomposition’ – the breaking down of professional work into component parts with IT solutions being applied to each part. 

They forecast the dissolution of expertise into many streamlined processes that will have the same effect on professional work as has already been inflicted on skilled occupations and craft skills. ‘Some of these parts will still require expert trusted advisors acting in traditional ways, but many other parts will be standardised or systemised or made available with online services’.

The outcome

I think you can see where I’m heading with this. If we discard the scientific basis that underpinned the development and evolution of market research, the opportunities to apply software solutions, such as selecting samples from access panels, administering surveys and analysing the outcomes within a single process, or a linked series of automated processes, becomes one example of decomposition. 

Using algorithmic based tools to analyse data, especially from social media (where there is unlikely to be any real form of systematic sampling applied to selecting data), is another example, where the need for any intervention by humans is already questioned. 

Some think that market research is simply a toolbox of craft skills, purchased to achieve a given end; others call it an industry, again with the implication that the expertise needed is more about skills training than creative problem solving; others argue the need for adding value through applying consultancy skills, perhaps moving us more towards the professions end of the spectrum. 

However, if we thought that a move in that direction was the salvation in protecting what we are and do as a counter to the effects of ‘decomposition’, Susskind’s argument implies that this offers no safe haven. 

It all comes back to disruptive technology. We’ve discussed the impact of this on market research for years, but are we any closer to identifying a strategy to carve out a future path for market research? 

In an earlier blog I discussed the need for market researchers to become fox-like in their outlook in today’s complex and contradictory world, a view echoed by Susskind who sees the survival strategy for professions being the creation of new services delivered in new ways. 

Dr Frank Shaw (Centre for Future Studies) thinks that: ‘Those professions that do not change will render themselves obsolete. Those that are able to transform themselves – and I mean “transform” – will thrive and prosper’. 

Are we up for that transformation, or will we become victims of decomposition – perhaps in some cases self-inflicted by rolling back the strengths that founded market research?