The Bad Moderator: why boring surveys are killing your research

Flood – 4 December 2024

Source LinkedIn

Picture this: you’ve just spent a significant chunk of your research budget on qualitative research and you’re behind the glass viewing a focus group.  But as you watch, something doesn’t feel quite right.  The moderator doesn’t really seem to be listening to what people are saying and is simply following a pre-defined script.  The respondents appear increasingly bored and some start looking at their phones, no longer paying attention to the questions.  The results from the research end up being pretty bland and uninspiring.

Would you use that moderator again?  Probably not! But here’s the thing - that’s exactly what’s happening in the world of online surveys.  The only difference is that the part of the process where the data is actually collected is out of sight, and so often overlooked.  If you care about research actually delivering something meaningful, that’s rather a big problem.

A data quality crisis?

It’s not exactly the world’s best kept secret that the quality of survey data is in decline.  Research Live is awash with articles on the subject going back several years and there was even a podcast series earlier this month about a data quality ‘crisis’.

So what’s going on?  Ask most researchers, and they might explain to you it’s all respondents’ fault – they’re fraudulent - doing surveys as fast as they can to get points, or lying to make sure they ‘qualify’ to take part.  Or perhaps they might even explain it’s a dreaded influx of AI bots.

Now don’t get me wrong – these are real issues.  But they’re pretty easy to spot and any decent agency will be cleaning them out anyway.  The real problem is more fundamental.  In another enlightening article from Mark Ritson last year, he referenced a study exploring this exact topic – what are the true causes of poor quality data?  The things mentioned above barely register.  The real menace, by a huge margin turned out to be something that’s pretty simple, but largely ignored: boring surveys.

Boring surveys

Let’s face it: most market research surveys are boring. They tend to mimic the format of their paper ancestors - endless rows of checkboxes, dropdowns, and scaled questions that force respondents into a passive, robotic mode of answering. This format was built for an analog world, yet here we are, decades into the digital age, still copying and pasting that experience online.  In the era of personalized, dynamic content and increasingly impressive technological capability this approach is woefully out of touch.

Yet our industry clings to the standardised survey format because it’s easy to administer and analyse. Unfortunately, this offers only a false sense of convenience. Just because a survey is easy to fill out doesn’t mean it’s effective. And just because you can run some fancy stats modelling on your 10-point scale, doesn’t make it meaningful.

Checkbox zombies

The current way most surveys look and feel is not just outdated, it’s engagement-killing. If your survey is a digital version of filling out a paper form, can we really expect participants to give us anything more than the bare minimum?  Nobody wants to spend 15 minutes clicking through redundant, irrelevant questions.  When people are bored, distracted, or irritated, you don’t get meaningful answers—you get checkbox zombies.

Here’s the thing - engagement isn’t just a matter of user experience; it directly impacts the quality of the data and resulting insights we can get. The industry has been living under this myth for too long: that we can serve up tedious surveys and expect accurate responses in return.  Unengaged respondents give disengaged answers, which lead to insights that are either shallow or outright wrong.  For research buyers, the implications are serious.  Strategic decisions based on disengaged responses can lead to misaligned product development, ineffective marketing campaigns, and missed opportunities.

Where have all the fish gone?

Boring surveys create a vicious cycle that makes the problem progressively worse.  People are now so inundated with tedious surveys, many have simply given up doing them altogether.

This can be easy to ignore when you’re using panels.  There’s plenty more fish in the ocean, right?  Well, from an agency perspective the signs aren’t great.  Audiences that were fairly easy to get hold of just a few years ago are increasingly harder to reach – and we know that most panels have long since resorted to ‘topping up’ from their competitors.

The situation appears even worse with surveys sent directly to customers – we’ve heard from many organisations that fewer and fewer customers are taking the time to fill out their surveys.  And the ones that do are increasingly negative – those angry enough to take the opportunity to have a moan.

Boring surveys are like bottom trawlers, making the environment so barren that most of the fish have gone – and the ones left are less diverse.  Not only is it getting harder to get a robust sample size, but our perspective is becoming significantly narrower.

Engagement is the solution

If we want true, meaningful insights, we need to move beyond the paper-form mentality.  At Flood, we’ve completely reimagined what surveys can be – it’s less like filling out a tax form and more akin to a human conversation. We've ripped up the rule book to make surveys visual, interactive, tailored, and yes - even fun.

What we’ve found is that this doesn’t just make the process more enjoyable; it keeps respondents focused and thoughtful.  When we properly invest in engaging our audience, they reward us with their time, attention, and richer, more thoughtful answers.  Meaning we can develop more nuanced and valuable insights.

This difference was experience in a striking way by a global brand we recently worked with. After years of traditional surveys showing their product was meeting consumer needs, our engaging approach revealed a critical issue impacting purchase consideration – one that conventional checkbox surveys had missed entirely. This insight led to a successful shift in marketing strategy that resulted in a significant uplift in sales.

So, when you commission your next research project, make sure you ask to see the survey.  Then ask yourself: is this something you’d want to fill out? If the answer is no, your respondents probably feel the same way.