NEXT 2019 Podcast Series

NEXT 2019 Pre-Conference Series – Frank Kelly – Ipsos

The 2019 NEXT pre-conference series is giving listeners an inside look into companies such as IBM, Voice Metrics, Ipsos, and Pulse Labs.. Join insight leaders on June 13 – 14 in Chicago for NEXT, where you can discover how technology and innovation are changing the market research industry. In this episode, Jamin Brazil interviews Frank Kelly, Global Head of Operational Product at Ipsos.

Find Frank Online:




Hi, I’m Jamie Brazil, and you’re listening to the Happy Market Research podcast. This is a special episode connected to the Insights Association’s NEXT conference, which is being held in Chicago on June 13th and 14th.

My guest today is Frank Kelly, head of Innovation and NPD at Ipsos. Now, Frank, I do have a question. I have always said Ipsos. I think it’s pronounced “Ipso”, is that correct?


If you are French.




For everybody else, “Ipsos” works fine.


Perfect. So Ipsos is a global market research and consulting firm headquartered in Paris. Founded in 1975, Ipsos is publicly traded and ranked in the world’s largest market research agencies, actually number three, I believe, with offices in 88 countries and employing over 16,000 people. Prior to joining Ipsos, Frank has held senior leadership roles at Nielsen, Greenfield Online, TNS and Lightspeed/GMI. Frank, thanks so much for joining me on the Happy Market Research podcast.


Ah, thank you very much.


You’re speaking at this year’s NEXT event on how to integrate voice in your total customer experience. My second episode on this particular podcast was centric to voice. I am so excited to see a conference, the first conference that I know of in our space, that is really kind of centralizing the communication around this particular medium. How did you first come to realize that voice was important?


If I go back maybe 8 or 10 years ago, when I started seeing people use voice to text, it dawned on me. As soon as people started using features like that, there’s this bound to be an application and research, because they’re just showing preference for a way to communicate. And we have to accommodate those preferences in the way that we capture data. But I guess the real big thing was certainly when Siri was introduced by Apple –I think it was 2011. That really seemed to show the promise of what you can do with voice communication. And it showed that with work eventually that could become a major component of how we collect research data.


Yes, for sure. You know, it’s interesting 2011. I think it’s September 2016 that Alexa launched. I think I have the year right. Does that sound right to you?


That sounds right. Yes.


So the big head start that Apple had inside of this space, and yet they certainly have taken a back seat from a growth perspective with Google Home now being the fastest growing voice-based platform. And obviously, Alexa. I think Alexa is still dominant.


Yes, well, again, a lot of people are using the Apple, the Siri on their other devices. So it’s not just the voice assistant devices that we should think about because research could be captured probably more commonly through the voice assistants on the phone.


Right. Yes, yes, for sure. I was talking to my kids. I have a bunch of kids, some older ones and some younger ones. So the younger ones are utilizing, and this is all just organic, Alexa right now to play hide and seek. It’s just really funny. Oh, Alexa, stop! Sorry. She’s now going to be called “A”. No, no, stop!

So the other side of it is, my older kids were talking about how it would be really cool if they could get an audio feed that highlighted stuff coming out of their Instagram account that they could then somehow magically connect right to their phones, to get in the visual pieces there as well. It is almost like a morning notification or update or what have you. So it’s funny how you think of them and “hide and go seek”, which is very much a tangible, visual-based game can be and has been created in a voice-only context. The other piece that I think is interesting… You mentioned Siri in 2011. I got my daughter an iPhone. She was 10 years old at the time, which I know it’s early, but don’t judge, and we’re driving, and I asked her: “Let’s talk about your best friends.” That sort of dad conversation, right? She said it was Hannah, another girl named Emma and Siri. She is actually pretty clever, my daughter. At least, I’d like to think so. But anyways, she actually told me in the car that, she was all in, that Siri was a real human being and just happens to be constantly available to answer questions that she has. And of the top people in her life, one was an AI.


Wow, that’s an interesting story. It is true, there’s people that are interacting with these devices constantly, many times during the day and they’ve customized the voice to their particular accent, to their gender, to their favorite sports player, whatever might be.




And it becomes a pretty indispensable component of their day-to-day life.


Yeah, for sure. And I think that as we pull back the user experience, and I’m not asserting your age on the podcast, but I didn’t grow up with social media by any stretch of the imagination. Libraries were the source of information for me, not the Internet growing up, and so my context is completely different from my kids. My teens, who have grown up with access to social media during pretty much their whole awake life. And then now my youngest ones, my two and three year old, who are interacting yet a whole different way with technology and access to entertainment and information. As leaders, we need to make sure that we have, that at the industry level we create the humility to understand that here’s a massive evolution that’s happening in these other two generations that we just don’t exactly connect with.


Yes, that concept came out a while back about the digital natives and so forth and much what we call the youngest generation because they are even more native. But it does also suggest that across the other generations, there are some that maybe are much less amenable to these technologies. And so it means that when we create research services, we have to really cater to quite a range of capabilities in the technology space.

In many cases we have to use multiple modes of data collection, which each have their challenges, and they may be well suited for a certain generation and not so well suited for another generation. So you have to have a secondary methodology that you have to try, and make the two methodologies fit together somehow in order to capture the representative data.


It’s kind of like an analogy: being we went from paper to Internet, this whole transition in market research. Obviously, there’s still some surveys that are paper-based but predominantly Internet is taking that space over. And then, later on, in 2006, the release of the iPhone smartphone. So now you’ve seen, from 2006 to 2010 again a migration. I think the majority of surveys now, over 50%, are taken on smart devices. Are you seeing voice is part of that narrative?


I could talk about several different waves of innovation, and how researchers have followed the adoption, which was curved from… I worked in postal. To go back to those era. I worked in postal and CATI and face-to-face. And so most of those have transitioned to the online world, and then with mobile coming in, moving from computers to other devices, tablets for a while as well. But they seemed to be phasing out, but now the majority of collection is now on mobile, indeed. And within these devices, such as with mobile, with new technologies being available to move to voice, this will be an equal challenge and an equal opportunity, shall we say, to capture data in a new way. And like the other challenges, we made some big mistakes. When we first moved to the online, everybody’s heard the story about you just took CATI surveys and telephone surveys, and adapted them to the online, but didn’t really make use of the unique characteristics available online to the extent we could have. And then the same story was really true, it took a very long time to rethink how our surveys could best fit in the mobile environment. And it’s really just getting to that point now. And I’d like to think that at some point we learned from all these lessons and say:  “All right, well, you know, there’s going to be a sizable portion of data collection in the future, which will be voice-based. And let’s plan for that. Let’s figure out how to leverage that new methodology to its fullest from the very beginning and not kind of drag our feet”, and so forth.

I am definitely seeing enough of these transitions to see that this is going to be a big one. And it’s going to be an important one that’s going to provide a lot of great opportunities. And ultimately, there are going to be companies that do a good job of figuring out how to work in that new space and how to leverage the new capabilities of voice for data collection and there will be others that don’t, and suffer as a result of it.


Yeah, gosh, your point about mobile is really interesting to me. I hadn’t actually considered that, but, 2006, it is about 13 years later, and what you just said is actually very accurate. Actually, I don’t even know if we, as an industry, have completely adopted it. We just fielded a survey that was not mobile compatible, hilariously enough, we helped to fulfill sample in a survey I should say, and we saw a massive dropout rate. We wondered: “Why in hell is the incidence so bad?” It turns out we checked it on our smartphones, and sure enough, we couldn’t take the survey on a smartphone.


There’s still probably about 25% of surveys that can’t be taken on a mobile device. Something like that. It’s still a problem, you know?


Yeah, only if you want representation.


Now, the problem is that the people that are only doing it on a computer are not necessarily representative because there are whole groups of people that almost never worked on a computer.


Ha! That is exactly right. Such a great point. You know, that’s another interesting point there too, and I know we’re a little bit off topic, but is Rogier Verhulst, Head of Insights for LinkedIn, told me the day of the email solicitations is dead, which he’s making more of a point, not like a state of truth. His point is that there’s entire working parts of the organization that literally don’t check e-mail anymore right. They are using Skype, or sorry, well they are using Skype, but they are also using Slack and other platforms.


Yes, Slack.


Exactly. Right. Exactly. So if you’re exclusively soliciting feedback through email, which worked great prior, then you may be missing access to a subset of the population.


Yeah, well, on that point, we see ourselves soliciting responses from respondents through at least a handful of different ways. This in-stream, which is sort of in social media or in-Linked stream there, where you are recruiting people in real time to a survey.  In reality, in augmented reality. And there’s quite a bit of this being done there in social media, in voice, as well as the typical email and other communications methods.


So what is the biggest challenge for market researchers to get consumer opinions in a voice context?


Well, I would say that we can’t underplay the technology challenge of the AI component. What we really need to have is a conversation with respondents and conversation between a computer and a person. And to do that, the computer has to understand what the person is saying and use that understanding to then ask further probing questions. It’s one thing to just say: “Okay, can we take these basic closed-end questions, and get a computer voice assistant to understand them?” And I’d say: “Yes, We are pretty well there on that part.” But that is back to the point earlier. That’s just taking mistakes of, using what worked online and trying it on the next mode when there’s an opportunity to do better than that, and that’s through really moving to a more conversational mode in the voice context. And to do that, you really have to be able to understand what people are saying. And I think there’s this great progress being made in this area, that you can train a database at a category level, you can take social media data, for example, and use that to train a research database, which is much smaller, not adequate for training a database on the terminology used and so forth. But we’re not really there yet for large scale conversations, shall we say. But that’s where the ultimate goal is to me. As we progress with AI and with the natural language processing, we’ll get better and better over time at being able to make sense of what people are telling us. And based on that, asking the right questions to follow up on that.


So in 2023 it is projected that about $80 billion dollar will be done on voice, which is obviously a material channel, for whether it’s Walmart or Amazon or whatever, Google, etc. And that’s around the corner, so to speak, when you’re 48 like I am, it feels like it is pretty close. Why do you think voice isn’t a bigger deal in context of insights right now?


Well, it is getting a lot of voice, a lot of mentions at the moment. It’s got even more buzz. And, like I say, augmented reality that I just mentioned earlier is probably an even bigger industry than voice is today. And yet it’s not that important in a research perspective. Part of it is that I think research companies are pretty good at adopting, and using new technologies, but we’re not that great at developing them. So we need other industries that come up with these new technologies to come around and come into our industry, and help us figure out how to best deploy those technologies. The big industries for voice are things like automotive, or security, or financial, or retail. They are the early movers. I think research is a bit smaller, and the technology companies will be focusing on research as an opportunity in the near future. But the challenge is here. Technology is a bit more too complex for most research technology companies to try and tackle and optimize for research purposes.


To your point, do you see that as more of a partnership opportunity for a company like Ipsos? Or is it a hybrid where you’re going to be developing your own suite of solutions?


Well, the industry tools are going to be pretty good. I don’t know that many research companies are going to be able to tackle, to partner with the likes of Apple and Google and such realistically. But they’re making good tools available to be used. We have already been doing stuff within Amazon and so forth where you could get a skill picked up by those applications so that you can get surveys in there. And we’re going to have at the NEXT conference some good demonstrations. Myself and my co-presenters will be showing several examples of real projects that we’ve run this way. I think there are tools that we can leverage. I think that they will be building blocks that people will build upon because again, we need wide availability of these tools within whether it’s Siri, or the Google voice Assistant, and so forth. We need these things technology to be deployed out there and then we just have to make use of it.


Yeah, right. The analogy being for me, if you look at whether it’s Facebook Ads that have A/B testing baked in or Google Analytics, obviously, which is to show you a nice point of view of what’s happening in your website or apps, one of the things that market research is doing a good job of right now at the brand level is tethering that data to state consumer opinion data, and then creating more context for the business insight. It sounds like what you’re saying is voice is going to follow a similar suit where you’ve got these building blocks, whether it’s AWS or what have you, that will empower the bulk of the platforms, and then specialty pieces whether it’s an injection of the lady whose name starts with “A”, who I don’t want to be in on the interview, she could follow up with the: “What do you think about that last purchase?” That sort of structure.


Yes. I would say that we don’t need to create everything from scratch. We just need these tools. Everybody has a mobile phone. Many people have voice assistants. We just need to leverage the technology that already exists within them. And then, we can layer on other technology. In the case of the voice assistant, you’re not getting actual voice, you are getting the transcription of the voice, but I think Video Analytics is on the rise. And I think that’s going to be a big type of voice data capture that will be very important to the industry. Because I do think that the sentiment, emotion and voice tonality and all those different things that you can layer on there can further help you understand your research participants. So I think that down the road, I could see us doing segmentations based on voice, based on what we teased out of a voice in terms of the personality types, and they would be equally as valid as an answer in a questionnaire.


I think you actually already answered this in a number of different ways, but I’m going to really try to kind of hone in on this practical point. What is one practical take-away that participants can get out of your talk at NEXT?


I’m sort of a field work expert, if you will, I have been doing it for a long time. And so what I tend to look at is… And I know that researchers are going to ask a lot of questions about representivity, and how you blend sample, how you adjust the data for different collection modes and those types of things. So I am going to try and at least get some initial answers to some of those researcher-type questions in addition to the use cases that we hope to have several, like I said, good use cases that will illustrate the best ways to leverage functionality.

One thing I didn’t really explain is that like in a diary situation, there is a lot of repetitiveness in the data collection. You don’t want to be asked the same question again, and again, and again. If you have a series of five questions, you need to answer in a diary several times a day every time you do something like the laundry or something, using the AI techniques you can just give an answer for all four questions at once. And the voice assistant will listen to it, think about it, and recognize that you have answered all the questions and just say: “Thank you”. So there are some practical uses like that, that do make the research experience better for the participants, and that’s really what we’re focusing on the near term.


Yeah, I think so. I’m really excited about the point you just made, which is we care about the respondent experience. Research is right now being done at a scale that is unprecedented. Even if you look back 3 years ago, it’s crazy, but even if you look back 10 years ago, or 20, it is mind boggling that we’re doing as many surveys as we are right now. Everybody is doing surveys, it feels like. I got my tires changed the other day. And guess what? The local tire shop, which is not like a national chain, sent me a follow-up NPS survey, which was hilarious. So they are really starting to care. Seeing market research as an extension of the brand is important. And so we need to be better stewards of the respondent. I go back to the eHarmony example. I don’t know if you are familiar with that product.


Yes, not directly but yes.


Right. Prior to when I got married I did use them, by the way, but it was.. it was arduous because it was right when they were relatively new, and it was about a 400 question survey. It was really, really tough, like a multi-day thing. And they were very early in the space. They were able to reduce it to a subset of open ends with just a few closed-end questions and then, from the open-ends did populate the 400 variables that they needed in order to optimize matches. So the interesting part there for me is that they did this in a text-based environment, early leaders in sentiment and such but now you think about a 30-minute survey, and it is really hard to do that. I would argue, nobody could actually do that, maybe 0.25%. But you could actually have a conversation and have that be meaningful and then ultimately populate the data set in a way that can be analyzed for researchers.


Yeah, that’s very much my point of view. I think about qualitative and quant. They are coming together, and I think it’s a good thing because quant people are good at listening. And quant people are really experts at asking questions. But they tend to ask questions with a set of closed-end responses. And what we really need is good questions with open-ended responses where people can say whatever they want, and people that say some things that if we find particularly interesting, maybe we go back and do a return to sample through a voice survey or some other method. It might make it tougher for Data Analytics because it’s less structure data but ultimately, again, it’s getting more towards a conversation, person to computer, where each conversation eventually becomes unique. That is driven by the participant, not by the researcher. That is when we really know we have come full circle from the original days of face-to-face and CATI where it was a conversation as well. But then we put a lot of technology in between and methodologies that, in some cases, made it easier and more efficient for us, but didn’t necessarily preserve the deep insights that we were trying to get. So ultimately, if we can find a way to do things fast and at scale with deep insights, then we have really succeeded. And I think voice is going to be one of those ways that is going to help us get there.


Oh my God. Yeah, for sure. I’ve been seeing this is a growing trend. I was just at IIeX in Austin. There’s a lot of qualitative technology companies that have been entering our world over the last couple of years and really, if you think about it, a survey is just a surrogate for a conversation, it just enables a conversation at scale. And the reason that we have closed-ended questions is because I’m lazy to analyze the data… Through AI what we’re able to do in the sense of analysis, etc., natural language processing, is really, for the first time, be able to have that conversation at scale so that we can get qualitative and quantitative or qualitative insights at quantitative numbers.


Yes. And so what you can do is when you ask that question, you can get a set of responses, and then you can do probing, you can teach the AI to know what to ask.

If they say this, then ask that. It gets smarter and smarter when you train the database at asking what is the next question to ask to get deep behind the meaning of what they have in mind when they answer a question a certain way. And that’s just very hard to do in a quant survey today with the typical survey technology.


So looking forward, not too far, but relatively near term, what do you think is next for voice powered surveys?


Well, let’s see. The real challenge is, of course, that surveys are too long. You really can’t do a 20-minute voice survey. It’s just not going to work. It really has to be more like 3 or 5 minute survey because people just don’t want to talk to the voice assistants in these long dialogues. So I do believe that there is more and more opportunities to break up those long surveys, and to look at them, like you were sort of suggesting, into pieces and ask different people different parts of the questions, infer some of the questions, and so forth. We have to be much more creative in how we approach surveys, asking people questions. Voice could be a very good component of that. But it has to be changed down to, using them for specific situations where you’re hoping to get their deep thinking involved, and then open it in that kind of environment to answer a tough question, not just something that is a simple tick box.  I think that there will be a place found for voice surveys very soon that’ll be just one of the tools in the box to capture research insights. And then over time, I think that we will just find more uses. But I think the initial uses will be returned to sample, like I mentioned, or diaries that every repetitive nature or very short surveys, which could go out for a very large portion of the population. And from that, you call out a smaller group that you want to do in more depth and maybe move them back to an online survey. So I think there’s going to be a lot more switch mode-type things and broken-up surveys into pieces, and a much wider range of interviewing techniques. Back to your IIeX example, certainly the tools around big qual have been coming out at a rapid pace over the last few years. There’s a lot of great stuff there to do more and more survey interviewing. And I think that we will be leveraging them in a voice environment fairly soon.


My guest today has been Frank Kelly, Head of Innovation and NPD at Ipsos. Frank, thanks so much for joining me on the Happy Market Research podcast.


Hey, thanks very much.


All of you, for more information on the Insights Association’s NEXT conference, again that’s June 13th and 14th of this year. Please visit That is  I hope to see there. Have a great rest of your day!