The meaning of research?
17 March 2014

Last year in an earlier blog I referred to a comment by Joan Bakewell published in ‘The Guardian’ some years ago (2003): 

"…surveys themselves need checking out. They are not value-neutral, laboratory-style findings. 

"They are often done in the street – in bad weather – by people with clipboards, who would prefer to be doing something else, with passers-by who do their best to avoid eye contact but relent before the supplicant’s ‘It will only take a minute'. 

"So why is this far from exact method used as reliable evidence for claims that can frighten us, threaten us and cajole us – but rarely reassure us?"

Two articles in the same issue of that same newspaper last week (6th March) shows that the sentiments underpinning this comment regularly come back to haunt us. 

Firstly, Suzanne Moore in her weekly column covers her recent experiences of ‘intimate marketing’, and the incessant requests for feedback following service calls, visits to websites etc etc, citing as the reason she has failed to respond to Ocado’s invitation to celebrate the anniversary of her first purchase from them online: 

"The thing is I haven’t paid this anniversary much attention as I am tied up with giving constant feedback to my mobile ‘phone company. Every time I query the bill, I get 27 texts asking me to rate the performance of the person I spoke to. Trouble is, I am still in the middle of rating the venue and bar facilities of a fringe theatre I went to at the weekend." 

How many customers bother to respond to these requests; how representative are those responses that are given; what happens to all this data, anyway; does it really make a difference to the service delivered? 

I admit I never bother, whether the ‘experience’ has been good or bad as my expectations of being listened to or anyone paying any attention to my ‘views’ are extremely low. Over familiarity without good service, is how Moore describes it, ‘the globalisation of etiquette premised on the American model’ part of ‘harassment marketing’. 

This trivialisation of market research should worry us, as I’m sure it contributes towards the steady fall in response rates we’ve seen over recent years. How to tell the genuine from the trivial?

The second example is much more worrying. We are constantly faced with a barrage of claims made in the name of medical advice, based on scientific research, that this or that food is bad for us. It’s too much sugar, salt, red meat, caffeine – a new worry every day, just the sort of issues that concerned Bakewell. 

But in the other article that day, ‘Deadly diet’, Zoe Williams claims that the latest research from the USA, using data from the national diet survey in the USA (NHANES) is wrong in highlighting a link between protein and premature death. The reason is that the NHANES data is based on what respondents tell you they eat. Williams cites Jack Winkler (a former professor of nutrition policy): 

"You ask people what they eat and what you get back is lies. People respond normatively. They put themselves in the best possible light. They’re not harmful, deceitful lies…..Underreporting they call it, but in plain English its lies." 

So, any correlation between food intake and premature death is suspect. Williams states that a study of NHANES has concluded that the data across its 39 year history was not physiologically plausible - if respondents had really ate what they say they’d eaten, in two thirds of cases they would already have died! Winkler says: 

"Academics sit in front of a computer screen and do data analysis. But they never question the primary data." 

I know the feeling; I see it regularly in submissions to IJMR that get immediate rejection. By the way, Winkler concludes that sugar is the big worry for 2014.

In both cases, it’s surely bad news for research to attract this sort of publicity. All news is not good news. 

In the case of the former example, maybe it would be good for organisations to publicise, with examples, the way customer feedback is used to improve service – to reassure us that participation is in fact worthwhile. 

In the second case, maybe NHAMES needs to justify its methods, or, we need to counter with examples to demonstrate that we have methods to collect data on topics such as food intake that are more reliable.

Where in all this is the reassurance that Bakewell asks us for?