Researchers need to talk about AI

“The future is already here – it’s just not evenly distributed” William Gibson, 2003

AI is here and is already being put to work to support market research and evidence driven decision making in business. At the same time there is also a huge sense of scepticism about its value to the Market Research community.

At Insightflow we are trying to unpick some of the contradictions inherent in those two statements. We are conducting a series of interviews with various experts in the fields of AI and Market Research with a hope that we can, at least, help structure the conversation.

In many senses this discussion is part of a broader conversation about if and where AI will impact any business sector. However, the Market Research sector has a number of characteristics that might make it a fertile marketplace for AI:

  • MR often has large or very large data sets available for analysis – Large data sets are very much a necessity for AI to thrive.
  • MR has a huge amount of human interpretation of data to understand evidence, create insights, and write the business stories that are the product of good research – This human activity provides the training data sets essential for the effective use of AI.
  • Market demands are creating a need for increasing research velocity – Business is asking for more evidence to be produced more quickly, driving the “…need to convert unstructured data to meaningful data and insight at speed and scale.”  – Brian Livell, Big Sofa Technologies

What can AI contribute?

Amid the many predictions about how and where AI might be brought into the MR world, there are a number of potential benefits that many of the community feel are plausible:

  • Reduce the dog work for individual researchers – AI will allow researchers more time to do what humans are good at. “Companies that can combine AI-based technology with expert consultancy will have significant advantages in the industry over those that fail to embrace AI as a driving force.” Brian Livell, Big Sofa Technologies
  • See and present patterns in larger data sets – as data sets grow, AI (when well trained) can rapidly provide structures and patterns that humans may be unable to see. In turn this may make previously unseen correlations visible. Although it will remain a human task to understand what those correlations mean.
  • Provide a link between quant research and qual research – Big Sofa Technologies describe this process as ‘TaPaaS’ – Technology and People as a Service. Throughout these conversations has been the theme of ‘humans do human’.  “In the bigger picture ‘pieces of data should be seen as incitements to excitement’. The implication is that the excitement comes from what humans can do with great data and in turn it’s great tools that can produce the input pieces of data.” Ben Smithwell

What are the issues with AI in research?

The fact that many in the MR community regard AI with some scepticism indicates that there are problems. The trick may be to understand those issues and bring that recognition into deciding how – and when – AI might be used.

Many people have insufficient understanding of both the potential and the limitations of AI. 

“People don’t understand well enough what it could deliver to specify what it should deliver”. Jason More, Chattering Monkey

The result is a lack of credibility for AI driven solutions. This is exacerbated by the fact that many practitioners have already had a bad experience with the gap between AI’s potential output and the reality of how projects have been delivered.

“Whenever I read that AI has been used [on data] I am a great deal more careful about how I use it” Adrian Rhodes, AR Consulting

There is also the hyperbole that comes with new technology entering a marketplace. Inevitably some of the claims are overblown. “Smart companies selling mediocre software to uneducated buyers.”


There is a risk that AI and the study of big data turns researchers from people driven by nuance and lateral thinking, to being people driven by ‘counting’ things. Almost everybody is aware of the growing links between AI and research. There are many evangelists who see it as, at the very least, a tool that will help researchers do more of the things humans are good at. On the other side there are plenty of sceptics who will point to the need for large training data sets, a continuing lack of accuracy/nuance in AI results, and the cost of implementation. There are also ethical, privacy and bias issues with AI that deserve closer attention.

It is a conversation that won’t go away. It feels like a conversation in which many researchers should be involved.


Our thanks go to Brian Livell from Big Sofa, Jason More from Chattering Monkey, Tim Deeson from Green Shoots Labs, Adrian Rhodes from AR Consulting and Ben Smithwell from Smithwell.

Related Articles