Firefish & Pinterest wins ESOMAR Gold Effectiveness Award 2023

FIREFISH – 23 September 2023

Source https://www.firefishgroup.com/article/esomar-research-effectiveness-award-2023

The Challenge

At Pinterest, their mission is to bring everyone the inspiration to create a life they love.

But prior to this research with Firefish, they had struggled to define a cross-functional and measurable definition of inspiration. The questions buzzing in their mind of what is it, truly, and how would you design an experience or a product or a campaign to ensure it delivered maximum inspiration? Their global brand and creator partners – as well as the team internally – were eager to learn this, too and that is where our story began.

Like any good researcher, it started with a little desk research. There were so many definitions of it…but one really helped us understand what we were dealing with:

“Inspiration propels a person from apathy to possibility, and transforms the way we perceive our own capabilities.” Scott Barry Kaufman, Harvard Business Review

This quote showed that it was a hard topic to pin down (excuse the pun) and it definitely wasn’t going to help answer the central question – can we measure inspiration? Can we help Pinterest assign an inspiration score to every piece of content? And can we measure inspiration in a truly scaled fashion?

The Solution

We knew this would require us to look at it from multiple angles, from purposely-chosen markets that are very different to one another.

• We started by speaking to a range of experts, psychologists, academics and content creators
• We ran 80 digital safaris on Pinterest and spoke to people in online qual communities
• We ran large scale quant with 12,000 to get human ratings across 15,000 pieces of content
• We used AI semiotic analysis to identify visual trends in inspirational content
• We blended this AI with the human data to begin to use machine learning to assign inspirational quotients to about 180,000 pieces of content.

Once we had done all that, the real work began to make it impactful, usable – and inspirational –  for all.  In the end, we pulled all this together into a simple model to help Pinterest deliver against their mission.

To get there, the first thing we had to do was to break down inspiration into more manageable concepts. Speaking to academic experts who have spent their careers studying inspiration, we learnt that inspiration is a two-part process of insight and movement.

Insight is the feeling of seeing something new of value in the world. And movement is the taking of steps toward that thing of value. However, movement does not necessarily equal action. It can be as simple as a change in emotion or a shift in perspective. Action is one part of the broader idea of movement. It’s when the two come together that a full inspiration journey can take place.

A key finding was that this process seems to be universal – it looked exactly the same across all 4, very different markets that we studied (USA, Germany, Brazil, Japan), across genders, and across generations.

While the inspiration process may be universal, inspirational content seems to be unique to each person. What inspires one person may not inspire another. However, there were very few images in our large-tested corpus that failed to inspire someone.

So this is how we landed on the ‘Inspiration Diamond’, with the Insight Phase being divergent – characterised by generative thinking. It is broad, expansive, and sees people being open to new ideas and information. In the movement phase, people are more consumed with convergent thinking – narrowing down those many ideas to those which are more relevant, and ultimately more likely to move from dream to reality. You may go through one phase faster than another, depending on your journey and goal but, typically, this is how it nets out.

Returning to the overarching method, our deep qual gave us a series of inspiration sub-dimensions in both the insight and movement phases, exploring each out in the world, and on Pinterest, uncovering what each phase means for content on Pinterest. We realised that that the sub-dimensions we were witnessing were not equals so, using quantitative surveys to explore each phase as part of an entire inspiration journey, we were able to assign weighting to each of the sub-dimensions to understand their contribution to the journey.

We asked people to rate images according to these statements (e.g. I like the composition, this catches my eye or attention, it looks authentic, genuine, or real, or the quality or resolution is good, it looks professional etc) to define the different weighting to predict overall inspiration of a piece of content.

We then blended this quant data with an AI-led semiotic tool in order to correlate user reported data with large-scale machine learning across 15,000 images to start teaching the machine learning what a sub-dimension actually looks like.

This work then led Pinterest to pilot their own ML predictive model that will tell them, their brand partners and creators how inspirational an image is likely to be. They continue to train this across verticals, with human evaluation to ensure it learns with human accuracy, with it having real potential for driving product impact even further.

The Impact

In terms of the impact and effectiveness of this partnership between Firefish and Pinterest, it was a linchpin in the creation of Pinterest’s long term vision and strategy.

It has been a critical jumping off point for additional research, including the aforementioned “inspiration algorithm” and an academic research partnership examining the links between inspiration and wellbeing.

The inspiration diamond is a foundational lesson for Pinterest Academy, our global learning platform for advertisers, teaching brands exactly how to inspire Pinners. And Pinterest’s CMO highlighted the impact of the research as part of our global ads summit event, ‘Pinterest Presents’ in 2022, which was viewed by tens of thousands of viewers, yielding tens of millions of dollars of attributed revenue within 90 days of the event, and hundreds of millions of touched revenue within 90 days of the event.

It shows that inspirational creative generates more value for Pinners, advertisers, and to the Pinterest business. Fundamentally, inspiring pins over-perform in both engagement and revenue. Inspiring stuff, we hope you agree.

The ESOMAR Research Effectiveness Award judging panel certainly did, bestowing it with the Gold Award at this year’s ESOMAR Congress in Amsterdam, with Joachim Brechta, Director General, saying this category was reserved for the pinnacle work in our industry. We will certainly take that.

With it having already won the Quirks Groundbreaking Research Award in 2022, we were quietly confident but, as you will see in the GIF below, there’s nothing quite like that feeling when you know you are about to take the Gold.

If anything about this work has resonated with your own, as you think about how to measure the unmeasurable or other such knotty questions, please get in touch with Firefish (or Daron directly) as we would love to talk more about pinning down difficult-to-define and hard-to-measure research topics.