NEXT 2019 Podcast Series

NEXT 2019 Conference Series – Zoë Dowling – FocusVision

Welcome to the 2019 NEXT Conference Series. Recorded live in Chicago, this series is bringing interviews straight to you from exhibitors and speakers at this year’s event. In this interview, host Jamin Brazil interviews Zoë Dowling, SVP of Research at FocusVision.

Find Zoë Online:




I’m Jamin Brazil, and you’re listening to the Happy Market Research Podcast.  We are live today at the NEXT Conference in Chicago. I have the wonderful Zoё with Focus Vision.  Zoё, how are you?   


I’m great, thank you.  How are you doing?  


I’m good.  When did you get in?


Late last night, later last night.    


Oh, OK.  Kind of late.


Kind of late.


Kind of late.  So, have you been to the NEXT Conference before?


Do you know I have not.  And I’m actually really excited to…  It feels like the agenda is a little bit different.  There’s a lot more focus on nuts and bolts. I’m speaking here with Ted Saunders and Roddy Knowles on Mobilize Me, which is research on research.  You don’t see that at conferences these days. Why not? This is important. And so, real excited because there’s other presentations like that as well.    


Yeah, totally.  You know it’s funny there’s an adjacent industry which is analyzing credit card transactions.  It’s a big, big, big space predominantly sold actually, sold into venture and PE and Wall Street firms so they can analyze like when there’s an issue with Chipotle, what its actual credit card transactions trending towards exactly.  What they’re doing is they release a bunch of research on research. And they’re constantly being quoted in Wall Street Journal, etc. It just truly amazes me that we, as researchers who have billions of transactions (not monetary), we don’t do a better job talking about that kind of stuff.    


Do you know from my perspective I think there’s been a trend away from that?  If you think five years ago, certainly ten years ago (of course, that puts us in the aging category) but this was all about web surveys, web data collection, mobile data collection, but also on the qualitative side, how do we create good engagement with online research community participants, and you just don’t see that now.  And, tell you what: we’ve not cracked the nut. It’s not a done deal.   


Not by a long shot.


We’re not doing amazing research, that kind of research on research.  But you’re right: there’s also a lot that we could dig in on the Big Data side.  Like what time of day? Thinking online communities, what time of day are people coming?  What’s the average? How do we aggregate that across projects? How do we then increase engagements?  How could we do some incentives? I think there’s a lot of different things we could do there to just make us better informed.  


Yeah, even something as basic as email-open rates over time.  I think with such an important topic… Rogier Verhulst with LinkedIn, he had mentioned that they’ve actually seen almost a complete decline, approaching a zero, with certain segments in the population just not accessing email anymore.  They’re utilizing a bunch of different tools to communicate. Before you and I because of my age…


That was tactful.


…I mean pre-social media, there wasn’t an alternative to…   You have like AIM or whatever (AOL kind of online messenger), but it wasn’t at scale.  And now, all of a sudden, there’s probably… I mean I could connect with you at least five different ways, not using email. 


And there’s different ways to get our attention.  And I think that’s interesting because if I think of both SMS but also think of push notifications ‘cause again thinking of them is a way to contact people that we are speaking to their participants.  That you see in the general population has gone up and down. And, far from SMS disappearing, it’s got its place but so does all the other messengers – WhatsApp, Facebook Messenger, and anything else you use – but then also push notifications.




Just know, am I just waiting to get something?  So I’m going to engage with you, but I just want that notification to come up on my phone at that particular time.  


Focus Vision:  what are you most excited about right now?


We have a really exciting innovation coming out that we’ve been doing a lot of work on for certainly the last year, the last 18 months.  We’re going to be talking a lot more about that in the coming months, certainly coming September. So I’m hyped ‘cause I think it’s moving the needle, and that’s what we all need to be doing.


Oh, I can’t wait to hear about that.  It’s in the qual space or quant space?


It is in qual but it kind of bridges both.


Got it, so qual at scale kind of a framework. 


Yeah, yeah, and it’s all in the analytical phase ‘cause that’s what we need, again I think as well.  We talk about better, faster, cheaper. And with AI and all of these machine learning and different types of tools that can help us get closer to that data, I don’t for one minute believe it’s going to replace our jobs ever, maybe.  


No way.  


Hundreds of years?


I mean not in the next maybe 50 years.  Some kind of like abstract construct.


And everything will change so much in a way that we can’t imagine, but certainly not now.  I think these tools are there to make us better at our jobs, to be able to get closer to the data quickly, and get some good truths from it.  I mean that’s what we need to do.  


Yeah, I mean we continue to hear data–rich, insights–poor.  And we have seen a proliferation of new tools and methods by which we can gather consumer insights, especially in the qual space, which has been really exciting over the last two years.  What we still aren’t doing a great job of is how in the world do I analyze that at scale. You have like individual fiefdoms where you’re seeing that kind of, but there still hasn’t been, from my point of view, a broader, holistic kind of approach.  You know, mTabs, but there again they’re really focused on the quant space. Anyway, yeah, I don’t know; we’ll see. I’m excited to see what you guys release.   


Yeah, and I think there’s a lot of scope from innovation.  And I’m sure there are a lot of companies that have this thing in the works just because analyzing at scale, making use of the data.  We’re always talking about this Big Data we have at our fingertips; it’s going to make us better and smarter. And so, yeah, it’s going to be good.  


I’m excited about your talk.  We’ll post a link to it if you write a blog post when this thing launches.  If somebody wants to get in contact with you, how would they do that?


Find me through a variety of methods, as you know.  Anything from LinkedIn but also email me at FocusVision,  Love to hear from people.  


Zoё, thanks for being on the Happy Market Research Podcast. 


Thank you for having me.


Everybody else, we’re live at the Insights Association.  Special thanks to the Insights Association for hosting us here on site as well as some of their other shows.  If you would like to learn more, check the show notes. Have a great rest of your day.

NEXT 2019 Podcast Series

NEXT 2019 Conference Series – Thomas Fandrich & Mike DeGagne – quantilope

Welcome to the 2019 NEXT Conference Series. Recorded live in Chicago, this series is bringing interviews straight to you from exhibitors and speakers at this year’s event. In this interview, host Jamin Brazil interviews Thomas Fandrich, managing director US and co-founder of quantilope; and Mike Degagne, head of sales at quantilope.

Find Thomas Online:



Find Mike Online:




You’re listening to the Happy Market Research Podcast; I’m Jamin.  And we have today some special guests. We are live at the NEXT Conference here in what is turning out to be a spectacular day outside although we are quarantined, I guess, in Chicago.  I’ve got Thomas and Mike with Quantilope. Guys, welcome.  

[00:22] – Tom     

Hi, Jamin.  Thanks for having us here.  Great to meet you.


It’s great to have you.  For just voice recognition, that is Tom.  


Hey, this is Mike from Quantilope.  


Yeah, you’re going to have to let him in.  

[00:43] – Tom   

We also have a lot of laughing.  So, now we can go to the serious part.  


I’m glad I called you here today.  Tell me what do you guys think about the show so far.

[00:50] – Tom      

Oh, we really like it since there are a lot of specialists here that have very serious knowledge about the industry and about market research.  It was great to get their perspective on a solution like ours, which is pretty much positioned on speed and substance, substance in terms of quantitative methods that go beyond simple service stuff.


Favorite session?  Have you guys been able to attend any sessions?  Mike, we’ll go with you on this one.

[01:20] – Mike

I have not; so, you should not go with me. 


Tom, have you attended any?

[01:26] – Tom

Me neither, so ah…


It’s hard when you’re exhibiting to go to the sessions ‘cause you feel like that’s the moment the big customer…  You know Coke is going to walk by your booth as soon as I go here. The head of insights for CES. It’s this whole like FOMO, fear of missing out thing that happens.  I’m suffering from it from it too, which is killing me, because I really wanted to hear the storytelling session, but anyway… That didn’t happen, and I also didn’t get Coke.  So, waah. Anyway.   

All right, so, tell me exactly what you guys do.

[01:59] – Tom 

All right, so Quantilope is an Agile Insights platform, right, that automizes all the single steps in a market research project, which is kind of finding the right tool to tackle your research question, program the questionnaire, and infuse it with high-end methodologies like System 1 Conjoint, MaxDiff, then field it to get the data in real time and get it visualized and analyzed in real time through our analyze module, and build beautiful dashboards of all that stuff to share it internally and externally.  So it covers the entire process very seamless and automized.  


Mike, you gave me a demo, didn’t you?  

[02:37] – Mike

I did not, but one of my guys did.  


One of your guys did.  Yeah, right, exactly. So, one of things that I thought was super interesting about the fundamentals of the platform is that I can execute in MaxDiff really, really easily.  So, I have my survey that is built out; that’s straightforward. Not really a big USB. But now, all of a sudden, I just add my attributes and it makes a recommendation on what types of statistical approaches I may want to use, incorporate into my survey design in order to answer my questions.  And then I also really like the reporting side of it (not that I’m trying to sell for you guys). But I personally was really impressed ‘cause it was all streamlined and easy to use. Tell me about some of the more popular applications.

[03:33] – Mike

So, ah, I actually started out in market research 15 years ago, and the way that I was doing studies back then (Conjoint, MaxDiff, Key Driver) was building the survey, taking the data out, cleaning it in SPSS, using Syntax.  And then I took it, put it into a stats package to get the Conjoint. And then I had to take that data out, cross-tab it. I had to use ETABS at the time to auto-populate the 140-slide tracker study for Microsoft, and that would take two months.  And then I would present it, and then they would want to change the segment. And I would cry…on the inside.


Yeah, yeah.  I bet you were happy cashing that check.

[04:10] – Mike   

Well, I was working for somebody else.


I know, I know.  Somebody else was happy they’re cashing the check.

[04:16] – Mike  

Yes, and I was just happy to have a job post-recession.  But with our tool, what’s really powerful is that it can do a MaxDiff for something like this, Conjoint, Van Westendorp in 1 to 3 days and it’s because it was thoughtfully put together.  It’s not a usage and attitude survey platform that we duct-taped or staple-gunned these advanced methods on. And so, the most common usage is going in, using our really advanced library of techniques and then things like MaxDiff, Conjoint, Kano, and then fielding it within minutes, and then having the data back in 1 to 3 days with beautiful visualization.  Did I answer your question?   


Yeah, perfectly.  So, it sounds like when you guys are looking at the customer utilization, what is the No. 1…  Are they using MaxDiff? Is that No. 1 or is it…?   

[05:04] – Tom

I mean this No. 1 thing…  It’s really the portfolio of methodologies that people appreciate, that they have the freedom to do MaxDiff today, System 1 emphasis approach tomorrow, and the Key Driver analysis yesterday.  They love the broadness and the flexibility our software platform gives them.  

I would add that this flexibility is when you have a business problem.  There’s a lot of people that get caught up in research where they only do MaxDiffs.  When you’re introducing a new product… We’re getting a lot questions about CBD oil in like food products.  So, what does the category mean? When people think of CBD oil, what does that mean for the category? And how does that actually relate to the product itself, the brand itself?  Then from there, you can go through a whole different litany of research approaches to actually get the product idea, the product price point, the product packaging, the product claim.  And so, that’s what we find, that our clients are working with us from ideation all the way to bringing the product to market.    


So, do you guys do services or are you just providing the platform?  

[06:09] – Mike

Yes, this is actually another unique identifier for us is that we have…  Tom runs a team. And I know Bea is listening right now; she has a PhD in neuroscience.  Vanessa has an advanced degree in analytics. So, we have this team that we call the genius bar where you can come to them and ask questions.  And so, if you’re a brand manager and you don’t have experience working with MaxDiff or Conjoint, you come with a problem and we show how you do it.     


So you’re supporting in the way of training on the tools and make recommendations to methodologies but not necessarily executing a project from A to Z.     

[06:41] – Mike 

We can.  The example that I give is it’s like getting into a Tesla for the first time.  You’re too afraid to turn on the autopilot yourself; so, someone else will sit with you and do it.  After a couple times you feel comfortable… 


Have you done the autopilot with a Tesla?  


I have, yeah.


So, the first time I did it, I was by myself on the freeway.  I’m not exaggerating: I thought there might be a 30% chance I’m going to run off the freeway.  I mean it’s that terrifying.


So you were white-knuckling it the whole time.


Literally.  I’m not going to let you control my life, Tesla.  Double tap and now I can’t stop. I’m completely attached.

[07:12] – Mike

Yep, exactly, but overcoming that fear, especially if it’s an advanced method that you don’t know a lot about, like Conjoint…  If you don’t understand the different product features, the price per pound or ounce, you really can create a Conjoint that doesn’t mean anything that someone with an advanced degree in analytics would be able to shoot holes in.  And so, we find that for the first 90 days, it’s a lot of training and hand-holding. And then soon we see these people that have upscaled their career by being able to learn how to do these things on their own.


So, thinking about that, you wouldn’t normally do a ever full factorial, right?  I’m thinking about like Conjoint. You know what I’m talking about? So you do a partial factorial.  So, are you able to then control how many cards respondents are ultimately answering for, which is kind of the trade-off of “Do I want partial data and then stitch it together?” or “Do I want a full…?  Everybody to see every iteration of the product feature set.”

[08:08] – Tom   

So, we are a ROCBC approach.  We are showing all attributes on the card.  But you don’t have to care about the efficiency of the design, the meaningfulness of the design.  This is just a click on a button. So we are optimizing the designs for you due to the efficiency criteria.


Got it, OK.  So you’re taking that into account already in an automated fashion.  

[08:30] – Tom

Yep, absolutely.


Got it.  Yeah, perfect.  Good. Who’s your ideal customer?  

[08:33] – Tom     

Our ideal customer has a lot of research needs and is very innovation-driven and tries to create new things, new products, new services, new advertisements; wants to explore new categories and all that stuff; so, is very active in terms of research; and is open to work with a platform like ours, right, which is super innovative as well and gives you the freedom to do a lot of the stuff maybe you have outsourced before to keep it under your own roof, to have full control and full transparency of all the steps in the process and all the data you are getting there.    


Mike, what is your favorite project?  

[09:20] – Mike

We did a Conjoint study.  There was a big CPG brand that got a call from a big box retailer, the big box retailer, that they were losing a losing a series of SKUs, and they were going to a generic in-house product and they had four days before they had to meet with the buyer at Walmart.  And they knew that they needed a more substantial approach to be able to make their point that this will hurt their consumers. And we were able to do it in that short turnaround time. That is an example of Agile Insights. So, the word “agile insights” is thrown around way too much today.  Agile insight is speed; there’s tons of free or nearly free speedy tools. But it’s speed and substance, and the substance is what creates agility in our opinion.  


So, talk to me about price point.  What does it look like? What’s the terms of trade when you do work with Quantilope?

[10:15] – Mike    

You know we are a SaaS company.  We’re a flat monthly-fee business, and it ranges from a few thousand dollars a month to tens of thousands of dollars a month, depending on are you running thousands of studies, are they all Conjoint. 


So, it’s like a price per study as opposed to per respondent or…?

[10:32] – Mike      

No, it’s actually access to the platform:  so, logins, trainings, genius bar, the types of studies.


Like basic something else pro or whatever.

[10:42] – Mike 

Yeah, and we’re flexible, right?  So, we understand what I think of as the SaaSification of market research.  The industry has worked on an ad hoc basis forever.


It’s hard to turn that corner, but it is actually…  It’s funny ‘cause it’s becoming more and more normalized.  You’re seeing an increase in acceptance of SaaS model.


Yeah, absolutely.  To start, we also understand that there are so many vendors out there that have burned partners, promising the world.


Totally.  That happens a lot.

[11:07] – Mike 

So we think crawl-walk-run is a good strategy.  So if you want to start out in a different way, we’re very, very flexible.  We’ll explain to you until I’m blue in the face why the SaaS model is the right model, but again we understand that there’s procurement teams; there’s way of doing business.  


If somebody wants to get in contact with you guys, how would they do that?

[11:27] – Tom   

Either shoot us a mail to or visit us on our website; there is a contact formula there.


Perfect.  And, of course, we’ll include that information in the show notes.  Tom, Mike, thank you both, oh, yep…

[11:40] – Mike   

We’ve got one more thing.  


One more thing.  Well, bring in on.

[11:42] – Mike

One more thing…


Everybody likes value; so, if you have two more things, that’s OK too.


You only get one.  We’re going to leave you one more, right? 


They got to call.  They can get the second one.


Yeah.  So, I’ll give you half a sentence.  No, I’m just kidding. So, for the people listening to the Happy Market Research Podcast, if you reach out to us and you reference this, we’ll give you a free trial of the platform, and for the best customers out there, we would actually do a free pilot project for them.  




Yep, but you have to mention Happy Market Research.


Oh, I love this.  High five right now.


There you go.


That is bad ass.  Thank you so much for bringing that value.  I tell you what: you know, Insights Nation, this is one of the big misses (and I don’t mean any disrespect from any other guest that I’ve ever had) but you actually have an audience that listens to this.  I’ve had FedEx reach out to me; I’ve had two dozen-ish brands that have, unsolicited, said, “Hey, thanks so much. I actually use this to help me with procurement, which is so funny because I never had that as a framework for why someone would tune into the show and actually don’t think that’s one of the core reasons why but I do think it’s an interesting by-product.  And so, it’s a great opportunity to be able to leverage like the audience and, if you have something valuable that you can pass on to them, to be able to do that. So thank you guys very much. I think that’s very generous of you. I actually do have one other question: Quantilope – I want to eat it ‘cause it sounds like cantaloupe and I like cantaloupe. How did you guys come up with the name?      

[13:14] – Mike

You’re going to have to reach out to us to get the ….


Oh, no!  I’ll not letting you go.

[13:21] – Tom

It’s a long story, but a nice story.


Is it, really?  Maybe some other time.


There’s a really nice story behind this.


All right, you guys.  Thank you so much for being on the Happy Market Research Podcast

[13:27] – Tom

Thank you for your time, Jamin.  I really enjoyed it.

[13:28] – Mike

Thank you!


Everybody else, if you found value in this show, please take the time to either rate it on the platform of your choice.  This particular episode, I would really appreciate it if you just took three seconds right now, screenshot it, share it on social media.  It’ll take maybe 120 seconds. This probably represents about an hour to two hours of production time. I would greatly appreciate it. Have a wonderful rest of your day.

NEXT 2019 Podcast Series

NEXT 2019 Conference Series – Simon Chadwick – Cambiar LLC

Welcome to the 2019 NEXT Conference Series. Recorded live in Chicago, this series is bringing interviews straight to you from exhibitors and speakers at this year’s event. In this interview, host Jamin Brazil interviews Simon Chadwick, managing partner of Cambiar LLC.

Find Simon Online:




I have Simon Chadwick, the legend, here live at the NEXT Conference here in Chicago.  Simon, how are you?


I’m good, thanks, Jamin.  How are you doing?


I’m doing OK.  Thank you for asking.  The weather has been a little bit cold, but not too bad.  A little rainy, oddly enough for Chicago this time of year.  But what are you going to do? We’re in a conference anyway so…    


It’s not snowing.


Not snowing, not freezing.  Totally wins, yeah, totally wins.  Actually, it might even be warm for Chicago standards.  I don’t know.


I don’t know.  All I think is that when it rains in Chicago, it’s just because the temperature is turning the snow into liquid.  


Exactly!  Totally true.  It’s a very cold city.


Anyway, I love this city


I do, too.  It’s actually my favorite city globally, but it would be hard for me to move here just because of the winters.  But I absolutely love Chicago. The people are super nice. It feels like a New York, you know like a Manhattan, London-ish type. 


And great architecture and, you know, the rivers, and…  yeah, tremendous, yeah.


It is great.  So, what do you think about the conference so far?


Well, from what I’ve seen so far, not bad at all.  I’m really looking forward to my colleague Lucy’s paper later this afternoon, which is “Who Killed Advertising Effectiveness?”  Murder at the manor. It’s a detective story, interactive.  


Of course, Lucy would bring just riveting bend to that subject.    


Right, right.


Or any subject.


Yeah, she’s a great speaker.  So I’m looking forward to that.  Looking forward to the comedy show tonight.  Are you going?


Yeah, I am, I am.  


That should be fun.  


That’s going to be great.  


Unfortunately, I don’t know about you, but every time I come to a conference, conference calls with China or with India crop up, and you’re in your room, you know, yeah.


Totally.  It’s ridiculous; it’s absolutely asinine, very frustrating, actually.  That just happened to me today, actually. And I missed an important person I was trying to connect with.  Anyway, that’s disappointing when that happens. So, the show’s actually really good. I’ve enjoyed… I’ve been able to attend three sessions or speakers, whatever.  They were super informative. I’ve been impressed with the quality of the content.   


Yeah, I was at the storytelling one, and I thought that she did a good job:  actually, made it easier for people to understand what they had to do in preparation and how to actually keep this story alive after the presentation.  It was very pragmatic and practical.


Yeah, I love the tactical application.  I’m a sucker for, if you put up a blog post and say, “Three Tips to…”, then I’m like, “OK,” because I’m going to read it.  I like the… I can pull something out of this and apply it today to my life and have an impact.


You know, we as humans are so funny whenever we see numbers.  Talking about Lucy, again, she did a big experiment at Coca-Cola Western Europe, which she presented at ESOMAR, looking at how you can disseminate insights to people who actually were not the stakeholders.  So, if you’re Coke sodas and you find something, how can you get to cross to the juices or waters because it might actually be really relevant to them. So they conducted a big experiment. And guess what one of the things that actually grabbed people was, in terms of the emails that you would send out, was an infographic with “43 Ways That You Need to Do This” or, you know.  


It’s crazy.  It’s funny. So, now I’m going to say something…  I’m going to contradict what I just said. I, historically, was very resistant to clicking on the three tips or whatever because I just felt it was so canned and market-ish. 


Click bait.


Yeah, it smelled like bullshit (pardon my French).  But I can’t help it now. The data’s right. It is what it is.  So I’m not going to resist the wagon anymore; I’m getting on it.  You know, getting on it. So, what is going on at Cambridge?


At Cambiar.  


No, no, sorry.


Cambridge, I haven’t been there for a while.  


Maybe soon.  


Well, it’s interesting.  So, there are three buckets to our business.  The first is strategic and operational consulting, mainly for research agencies.  That’s where we started off 15 years ago. We then added various different pieces of specialized operation consulting.  Then we got into the M&A business, and that’s now probably a third to a half of our business. And then we got into the training business on the corporate side in what we call Power Skills:  communication, influencing, synthesis, things like that. They’re about a third, a third, a third. But each one is showing different traits. On the client side the demand for training in Power Skills has shot through the roof.   




Really shot through the roof.  It’s quite amazing. We’re booked now through the year.  I mean we don’t have any space. And that’s with major banks and pharmaceutical companies and so on.  So that’s really good because it means we’re getting… We’re helping them achieve impact and a lot of it is actually about measuring that impact, which is also really good.  But then on the M&A side, it’s almost dried up completely in the States – nothing much left to buy, but it’s exploding in Asia. 






Who’s doing the buying in Asia?  Is it conglomerates?


It’s mainly the European and U.S. conglomerates.


Big boys, yeah.


Not the usual suspects:  the big four, apart from one.  But three of them are in trouble; so, their appetite is somewhat diminished.  But, yeah, the bigger guys.  


Do you see LatAm being like another…  I think that after Asia gets saturation, then… 


It should move to LatAm, but I think there’s some really interesting stuff going on in Africa and in the Middle East.


Yeah, I’d agree.


The Middle East, in particular, we’ve seen a big, big sort of bump up in activity.  And now, South Africa was always strong, but now we’re seeing East Africa and some of the stronger economies in West Africa really beginning to come on stream.


Thinking about ESOMAR’s numbers on 46-billion-dollar space that we’re in in market research, the representation in these other countries has been small relative to the overall spend.  But it feels to me, and I’m basing this off of conference attendance, I’m seeing a lot more people from these other markets attend conferences. And that is starting to speak to me in terms of how much people are caring globally about consumer opinion.  Honestly, I think we’re at this beginning of a J-curve up as an industry, which is really exciting.


Yeah, I think one of the things that we have seen now is we’re at the end of the Chicken Little era, you know, where the sky was falling.  And there is this acceptance that actually this is a time of huge opportunity. It just looks different.  


Totally.  When you think about that like user experience research, specifically in the Bay Area, I’m seeing this a lot.  You know Silicon Valley, you’ve got almost 10 to 1; so, for every market researcher, there’s 10 UX researchers.  It’s crazy. There’s been an explosion in that space. If you look at the hashtags market research combined with MRX on LinkedIn, the number of people that follow it, it’s about 350,000.  If you look at the number of people that follow user experience, it’s over 4 million.    




So, yeah.  But what’s interesting on a Venn diagram – and I’ve been doing this with a few other people – you look at the types of research that are done in both disciplines, there’s some crossover, a material amount of crossover, right?  I continue to wonder if we’re not – while everything is going to be scaling up over user data, we’re also not going at the same time start seeing… because these people sit in different parts of the org structure, not together.  So I wonder if we’re going to see like this, not chief research officer per se, But like somebody that’s responsible for consumer voice at the C level.  


You know it’s so funny because…  You’re just about old enough to remember the great Jack Michael, you know, his books on research.  And he was such a proponent of that idea 30 years ago. And it still hasn’t come around yet. Also, in the corporate clients that we serve, there’s this battle going on as to who are the holders of the customer voice.  Who are the holders of customer insights, of consumer insights? Is it the research function? Is it data analytics? If so, what type of data analytics ‘cause there’s about 40 of them? Is it finance? Nobody seems yet to have got the structure right, let alone put in that type of chief customer-voice officer or whatever you want to call it.  There’s chief CX officers, but that’s very specific.    


This is such an interesting deviant from our conversation but, or deviation from my normal conversation, that oversight layer becomes really important because you think about things like data governance, compliance.  It’s actually the same and as important in all those disciplines. So that needs to be part of the job description… as we build out this fantasy job description, that’s part of it. And then also just having that person at the table that has a point of view at a macro level and a micro level.  So, UX seems to be very micro: in order words, “I need a decision right now.” So I do my little whatever (qual or…). And then market research tends to be at a macro, it seems like: larger segmentation, price elasticity, things like that. It’s kind of like this really interesting power that can sit at the table along with the CEO, and I would imagine a lot of eyes would get diverted to that individual because, at least at the board meetings that I attend, that would be a very germane…, right?    


Actually, when we were down in Miami, I was talking to somebody down there who today is in data analytics but he was VP of insights or whatever at a major company.  He remembered being in a meeting with a CEO and there was the Head of Production there, the Head of Sales, and the Head of Marketing. And the CEO said to the Head of Marketing, “So how are things?”  And he said, “Terrific, absolutely terrific. They’re really going well.” And the Head of Sales said, “Ehr, we’re not yet really up there.” Head of Production said, “It’s really bad.” So the CEO turned to the VP of Insights and said, “Well, which is it?”       


That’s exactly it.  That should be a New Yorker cartoon.  


It really should, yeah.       


That’s perfect.            


I don’t know if we’ll ever see that role, but there’s too much competition. 


I think we will.  I really do. Just think about the amount of waste that’s happening right now from a training perspective.  If I just focus, like a Facebook, if I just focus on the UX division, there’s a bunch of people there and at varying levels of skills and experience.  And then I’ve got a whole different set inside of the market research of different people, right, inside of the market research. And so, never the two will meet if it is the case really. Just the training overhead could be reduced materially if you had a unified road map on these are the core…


That is one thing that we are seeing right now.  We’re seeing more and more, when we go in to train in these major organization, that it is not just researchers.  It’s researchers, data analysts, competitive intelligence, strategic, and they’re all coming together for these. Whereas we would be training, two years ago we’d be training 25 people in a room, now we’re training 125.  


Right, totally.  See that’s really interesting.  I think about technology has democratized access to the consumer, which is exciting but also terrifying because…  My favorite saying right now is, “Just ‘cause I have a scalpel doesn’t make me a surgeon.” And yet because I can launch a survey doesn’t necessarily mean that I should because the way that I ask a question is actually really important.  People who are seasoned researchers know how to do that well, and people that aren’t, they really don’t. And this isn’t necessarily intuitive. So the added educational level, again going back to that C-level executive or whatever, now becomes really important to start caring about the overall understanding of consumer insights at a…  Because everybody from the intern to the CEO is doing surveys.      


And the very fact that you can do them, like as you said, “If you’ve got a scalpel, you’re not necessarily a surgeon.”  The very fact that you can get up a template and put something on it and put it out to the field and collect that data doesn’t obviate the need for the fundamental principles, and the fundamental principles, I’m sorry, still count.  You know it bugs the hell out me that I was trained for two years before I was actually allowed anywhere near a customer. A lot of that was in statistics and in sampling. These days, that is not there. It’s there in some of the sample companies, in some of the suppliers, but it’s not there in the brand level.   


Right, well, I mean there was going back into the 90s at the brand level, there was structured mentorship program in a lot of the companies I worked with, and that doesn’t exist anymore.  


GM, General Mills was famous for being the university of research.  That went away. And then there’s another aspect to this which is…  And I’m probably going to upset a lot of people, 


Oh, I can’t wait.


I already published it, so…  We have to face that there’s a lot of data illiteracy in companies, in marketing, in brand at the senior management level.  People don’t really know how to read data. And if you’re just putting data in front of them or they’re collecting their own, there’s going to be some really bad mistakes made.  


I mean, yeah.  Again, getting back to corporate waste.  I think this is probably one of the biggest gross margin opportunities in front of organizations and also at an SG & A level.  So you think about how you could improve your sales and marketing right now. You got the whole lean marketing framework (A–B test the heck out of everything), but a big part of that also is understanding the customer “why” and informing, if B is performing better than A, why is that the case because maybe that’s really important.  And that is now in the spot; it’s starting to get injected into the marketing side of it. But now, all of a sudden, it’s like, “Who’s going to answer that question?” Is it market research or is it UX or is it marketing? You know what I mean? So, it continues to beget the consumer voice is important to a decision I need make, but now who owns that, kind of rolling it again back up.     


It’s like the consumer voice is going through a synthesizer and, you know, it comes out female on one synthesizer and a male on the other and a robot on another.


And the waste then is not just redundant research.  That is a waste but that’s not the bigger problem. The bigger problem is conflicting and wrong answers that are derived just because of errors in the methodology and skill of the analytics person, anyway.  So, as you look forward to 2020… Can you believe 2020 is…


Just around the corner.


Oh, my God.  I swear I thought I had a jetpack.  When I was a kid, you said “2020,” there was no question we had jetpacks.


Yes, right, right, right.  My wife is really disappointed that we don’t have jetpacks.  


Right.  I think that’s terrible.  I think that’s terrible. Like we haven’t had a material improvement to our air travel across the pond, right?  That’s the other thing I think is… The Condor was one: the supersonic jet or whatever.


The Concorde, yeah.


The Concorde, yeah, yeah.  That’s what it was. But that’s gone now.


And I never got onto that, unfortunately.


I didn’t either.  I was super disappointed.  That was one of the things that I would have liked to have been able to do.  By the time, I could afford it, it was gone. Dang it, anyway. So, you think forward:  what do you see as a macrotrend?  


I really do think the biggest macrotrend is going to continue to be on the client side.  And I think it’s going to be this push for strategic relevance and strategic impact. You know when we did the re-benchmark with Boston Consulting three or four years ago now, 20% of insights functions were strategic partners or better.  I mean it’s pretty pathetic: 2 out of 10. The good news was it was double what it had been eight years previously. And I’m on a crusade alongside those from GRBN, Andrew Cannon and others, to get that number up to 30, to 40, and to do so by making people measure their impact, making functions and we’re seeing more and more uptake.  And I think we’re almost at the tipping point where we’ll get to maybe 30%. That still leaves 70% out there still not doing what they should be doing and still a lot of waste, to your point. But if that’s the case, then I can see a much more holistic use of all the tools that we see around us. God knows, how many of these tools can you actually absorb?  It’s really hard. Maybe that will enable us to get to Steve Phillips’ idea of we’ll have more time for the strategic, we’ll have more time for thinking. I don’t know. Everything, technology has given us that opportunity, we’ve wasted it but…      


Zoe, no, it’s not Zoe.  I’m sorry. Z Johnson, she currently works at Microsoft.  She had an interesting take on this exact point, which is…  And she actually cites in her podcast, MR Explorer, these milestones in her career where she’s had like a technological breakthrough and “Wow,” all of a sudden, she could do a lot more.  But she says she’s actually still spending the exact same time. It’s just on the logistics. They’re doing a hell of a lot more of the research.    


Yeah, yeah.  Unfortunately, I think that’s always going to be true.  But I’m an optimist. 


Me, too, absolutely.  Let’s do that.  


Yes, let’s hope.


Absolutely.  If somebody wants to get in contact with you, Simon, how would they do that? 


They can do it through or just look me up on LinkedIn and send me an email.


My guest today has been Simon Chadwick, the famed influencer and I think patriarch really in a lot of ways at least currently of market research right now.  Thank you very much for joining me on the Happy Market Research Podcast.  


It has been a pleasure.  Thank you very much, Jamin.


For those of you that are listening, if you found value in this show, which I certainly did, I would love it if you would please screenshot it, share it on social.  Just takes a minute. These take hours for us to produce; so, we appreciate that trade of time. Enjoy the rest of your day. 

NEXT 2019 Podcast Series

NEXT 2019 Conference Series – Ray Fischer – Aha! Online Research

Welcome to the 2019 NEXT Conference Series. Recorded live in Chicago, this series is bringing interviews straight to you from exhibitors and speakers at this year’s event. In this interview, host Jamin Brazil interviews Ray Fischer, CEO and Founding Partner of Aha! Online Research.

Find Ray Online:




Aha! is the name of the company.  We are live at the Insights Association’s NEXT Conference 2019 in Chicago.  We are winding up Day 2. What time do you take off


I take off just after this interview.  I’m going to head down to the train station, do the old school Amtrak back to Detroit.  It’s going to be awesome.


Oh, that’s right.  Oh, yeah, yeah, yeah.  


Yeah, yeah.  I take the train, get some work done, get to watch a little “Billions” on the way back.


How long does that take?


It’s about four and a half hours.


OK.  That’s not bad.


I lose an hour going back.  I go from the Central to the East time zone.  It’s a little bit longer that way, but it’s a nice, comfortable, great way to go back, you know, Detroit to Chicago.     


Totally.  You don’t have like all the drama with air flight.  


It is so simple.  It’s incredible. I mean they don’t even check ID’s going on, which is weird to me, but…


I know.  It is old school.


It’s that simple.  It’s old school. It feels like a throw-back, 45, 50 years ago.  


Yeah, it does, it does.  I remember the old days of air traffic prior to 9/11.  It’s like show up. Anybody who’s at the gate…


Right, the train is kind of like that now, but I will tell you they haven’t changed the cars in 45 or 50 years.  A little dated, but it is kind of a cool, romantic sort of way to travel back and forth, when it’s not too far.


Is it packed?


It is usually pretty full, you know, because there’s a lot of, again, Detroiters coming to Chicago and back and forth, a lot of transplants both ways. 


Yep, totally.  So a highlight of the show?


Highlight of the show.  I saw a couple of really good talks.  I’d say the first highlight though… When I go to these shows, they become so fun.  When I look back five or six years ago when we were just launching, I didn’t know anybody at the shows.  So, it was basically me and my two partners, standing at our booth talking to each other, telling stories, trying to make it look like we’re having fruitful conversations at selling something.  Now, I can go to the shows (we didn’t exhibit at this one), I’m just walking around; I see people like you; I see people like a lot of friends, vendors, prospects, clients, etc. I make new friends.  It’s just a really, really fun way to immerse yourself more in the industry but, again, as time goes on, it just becomes kind of a community of friends.      


ROI on shows.  That’s interesting, right?  I think about that a lot. Like how do I maximize my return on… It’s expensive:  it’s $10,000; by the time you’re done, it’s $10,000. You’ve got a couple staff, and that doesn’t include the hard costs of time and money going into it.       


No doubt, no doubt.  


So, what do you think about that?  Do you guys do much exhibiting?  


You know, we do.  We do about four conferences a year where we exhibit and I speak at several as well.  So those definitely help ‘cause when you have the exhibit and the speaking engagement, as you know, that’s kind of a double whammy, double power-packed punch.  You know where you got the ability to talk to people who come by your booth after they saw it. So you really max out, especially if you get an earlier time slot.  If you’re the last guy on the last day, you don’t really get to maximize that. And the ROI: usually it’s like a couple of projects and you’re good to go. Smaller projects you’ve covered your investment but, again, as time has gone on and I’ve gotten more and more clients, I can walk around this and it saves me from having to make 20 flights to different cities to see these people.  Set stuff up, go meet with them, etc., and they’re all here in one place. This venue is great; the hotel’s cool; great location, obviously, Chicago; it’s sunny today. It’s absolute perfect.         


Yeah, no, today’s like a perfect Chicago day.  


And to answer more directly your question about the highlights of the show, couple great speeches.  All of them were really smart. I think of this like a high-end conference: very smart people, a lot of client-and-supplier combination speeches, which are great.  I saw one yesterday that was with a Burke, an IBM person, two super smart people. Joelle was the Burke person; I don’t remember who the IBM partner was in that. But the speech was about Artificial Intelligence, AI.  There’s a lot of acronyms flying around today. AI stuff: fascinating to hear the perspective of IBM talking about it. They’ve got Watson; they’ve got amazing stuff. And Burke has been a practitioner and user, probably more on the quant side of things, or in the larger scale data sets.  But the fascinating thing I heard – and you and I have talked about this quite a bit – is that it’s one of things where it’s emerging and we’re trying to apply it. But it’s not the end-all answer at this point in time. And the IBM person actually said you have to look at Artificial Intelligence right now as a baby.  It’s truly a baby. It’s really going to be a great, grown-up human being soon, but it truly is very much in its nascent state. It’s young. It doesn’t have the answers. It can’t write the report for you as a human would, which I love to hear. And that’s something that you and I have talked about before especially on the qual side of things.  Human data takes a human touch, and we’re small data too for the most part. 




It’s not thousands of people, saying thousands and thousands of things and having millions and millions of words to draw from.  So, while it does apply obviously to that to help sort data, you know on our side, we still need that human touch. That was really good.  And then I just saw a really good speech about innovation and rapid product development with Thor Ernstsson from Alpha, and Vivek Bedi from Northwestern Mutual Life.  Really engaging. They’re both really good speakers, and they had a tremendous rapport. You tell they’ve worked together and they’re friends. And they had a couple of funny jokes about going out and drinking and whatnot.     


Isn’t it nice when you…?  It’s such a relationship business.  It really is. And kind of getting back to my question about the ROI, the benefit of having a booth is it creates an anchor point for people to be able to connect with you.  And once you have those relationships established, it’s gets harder and harder to justify it because, to you earlier point, you just call them up and say, “Hey, are you going to be at the show?”  And they say, “Yes” or whatever. And then boom, you know what I mean? You schedule that stuff out accordingly.  


And their stuff was great.  So, they were engaging and everything, but it was a very easy presentation.  They weren’t just running charts at us. They almost turned it into a TED talk.  So, it was really much more like half a dozen slides and some nice storytelling.  


That was it.


Yeah, it’s a good lesson for me and really anybody in how to deliver in a 30-minute format at a show like this where you just kind of tell a story.  Have some great visual things to support it; make it a little bit more interesting from that standpoint. But to be able to just tell stories about how you’ve done project work or come to some successes with big organization stuff and innovation and rapidly getting to market and innovating quickly.  It was very cool. Those were two of the better ones. All of them were good that I did see, though. And I had some good dinners here too, so.


Well, now, did you buy or were you treated?   


I bought one night, and I was treated last night.  


Nice.  Who was it?


It was Focus Forward, Kim Harrison, Dave Pataki.  We were out at Maple & Ash, so a plug for Maple & Ash.  Fantastic restaurant. Have you been there?




Seriously, probably the best restaurant in town.


We went to a pizza place.  So, next event you’re going to go to?


Next event:  good question.  I know there is one in August, the CX.  I think Mark Michelson has his CX Talks here, and I’m going to speak at that one.    




That’s here in Chicago.


What are you speaking on?


I am going to talk about customer experience.  He’s got these like TED-Talk, 10-minute slots, which is going to be super fun.  So, you’re basically going to have a couple slides that might be somewhat visual support, but it’s going to be a quick storytelling thing.  It should be really high energy, quick hits, main message in a storytelling kind of way. So I’m excited to do that. Mark’s a good buddy, and I know of yours too. 


Yeah, yeah.


It’s just fun to support his organization and his go forward stuff.  And he did approve my speech. So we’re good.  


Safe travels home.


I appreciate that.  


Everybody else who’s listening, I hope you have a fantastic rest of your day.  Insights Association, thank you so much for hosting the Happy Market Research Podcast on site at this year’s NEXT conference.  Congratulations to everybody who was speaking and the winners of the startup competition. It was SightX, by the way.  


Oh, nice.  


I know I got …  I was really sad.


Tim Lawton, nice.


But that’s OK.  I’ll come back stronger; I always do. Have a wonderful rest of your day.


Good stuff. Thanks, Jamin.

NEXT 2019 Podcast Series

NEXT 2019 Conference Series – Michaela Mora – Relevant Insights

Welcome to the 2019 NEXT Conference Series. Recorded live in Chicago, this series is bringing interviews straight to you from exhibitors and speakers at this year’s event. In this interview, host Jamin Brazil interviews Michaela Mora, president of Relevant Insights

Find Michaela Online:




My guest today is Michaela Mora, President of Relevant Insights.  Those are my favorite kind of insights, by the way.  


Yes, they are not any insights.  They are relevant.


I love that.  So, what do you think about the conference where Day 1 has ended at the NEXT Conference in Chicago?  Tell me what you think.    


Well, this is my first time coming to this conference.  This year I have been to several conferences, and I see some subjects coming up on a repeat.  It’s a lot about video and capturing data, more unstructured data and focus on tools at the same time figuring out how can we capture the full consumer or the full participant in many different areas.  And usually, the challenge of that is at the back-end. Many tools are very good at capturing the front-end. Then you go home and then you have to analyze.


Do the work.


Yes.  And that’s where still…  I test a lot of tools and there’s still some time to go.   


Does one session stand out as your favorite?


I really liked the one from “Who Murdered Advertising Effectiveness?”  Was a very good presentation, different style. 


“Who Murdered Advertising Effectiveness?”  That was a very sassy title, wasn’t it? 


Yes, it is.


That was Lucy, the speaker.  She’s from Keen as Mustard, I believe.


Yes.  Actually, it is a way to tell the story, to discuss a subject about elements that are affecting our industry.  It’s about changing times, and who is behind the changes and how we’re all involved in that. And so, that was a different way of presenting and discussing market research issues.     


Tell me about Relevant Insights.  What do you guys do?


Well, we try to help clients to make profitable decisions.  It’s about finding the right questions so that we can come back and help the clients to make the decisions.  So we do both qualitative and quantitative, both in the more traditional research but also in the user experience, user research methods.


Are seeing an increase in user experience?


Oh, absolutely, absolutely.  There is these conversions between the customer experience field and the user experience.  They come from two different branches of research. The customer experience, for me, is just a renaming of the traditional customer loyalty, voice of the customer field. 


It totally feels like VoC, market research, yeah.


Yeah, but the user research comes more from human factors.


Yep, products…  


Usability, interaction, ergonomics.  And in some companies, they are used interchangeably.  If you look at it as different focus on different areas of the customer journey, essentially…  Because the user experience right now has been mainly focused on how users are interacting whether on your website, your application but is going more towards the product and in a little more task base how you use that.  But that also has been an area where customer experience has come in like product testing, concept testing. So it’s a lot of mixed terms.     


One of the things that I find really interesting is it feels like a lot of the work that is actually being done between by a market researcher, which is my background, and user experience is stuff like…  I mean in 2001 or 2002, I was on-site at Intuit in Mountain View, California. They had built usability labs, and we were doing eye-tracking and exercises where we would say, “Can you do this?”, “Show me how you do this.”, with users, which all fits underneath this user experience umbrella now.  And it’s interesting, again, how… Like you said it, it’s funny I never connected VoC, but you’re right. So, it’s like market research, and then, all of a sudden, you have this variant that came out of it, a child called VoC, voice of customer, that was very big. And it somehow was perceived as something different than market research.  Like NPS almost holds a different space now than market research, right? It totally does. And now user experience, I think it has a lot to do with where the butt is in the corporation. So, if they’re sitting right beside product, then they’re user experience; and if they’re not, they’re probably market research or data scientists or…, right?  It’s an interesting kind of evolution.   


In some companies, customer experience is a more broad term, and then user experience is just one branch of it.  And others are going the other way around where you have the user experience. In terms of the language, if you’re a customer, you’re a user; if you’re a user, you’re a customer.  So it’s hard…  




It’s hard to make that differentiation.  So, that’s why has to do with where researchers have put their focus on the customer journey.  So, when you go to the interaction area, you do a lot of usability testing. Before I started the company in 2007, I was Director of Research for Blockbuster online, and we also had a lab.  And I was at before that and also we had a lab. It was very much into the web interaction. Now with more apps out there, it’s also extending to apps. But it’s about usability in many ways.  And there are quant and qual methods in that; there is information architecture testing in that. But it’s all about the interaction. But, at the same time, the customer is one. And so, you start extending that: “OK, I interacted with your website or your app, but I’m also interacting with customer service, and I’m also looking at your advertising.  And so, it’s part of this user experience, customer experience, maybe before and after you become a customer. How you call that, right? So that’s why it’s probably better to… Like I look at it more like the journey. When you start mapping the journey of the potential customer or current customer, now you can see how the different types of research going to that journey.  So we do both: we do both the traditional customer experience-type of research and also user research where iTracking, task-based, usability, and all that good stuff.  


Sorry, my voice is going away.  I’ve been doing this all day. It doesn’t like me talking, I guess.  Relevant Insights: Who is your ideal customer?


My ideal customer is the customer who understand research.  Right?  


Exactly.  Mine too.


That’s the one you want, right?  Because we have the challenge now in the last few years, which I have seen evolution in the industries, as new technology has come and facilitate do-yourself-type of research…  The reason influx of people who really don’t have a lot of experience in research because they have the impression that you have the tool, you can do it; anybody can do it. 


I have this saying I’ve been using a lot lately: “Just ‘cause I have a scalpel, doesn’t mean I should do surgery.


Exactly, but I was once working on a proposal with a team of people, different companies; there was someone – I think they were in an PR company – they say…  And so, we were discussing price and they couldn’t understand why it would cost so much. They said literally, “But isn’t that like copy-and-paste in SurveyMonkey?”  That’s the idea of how research is: just a bunch of surveys and you just come up with any questions and you put out there. And there’s no understanding on how much work goes into doing, into designing good surveys and to doing good analytics and all that.  And so, that’s the challenge that, as technology has facilitated make it faster and easier to do, then also it is attention for quality, right? Because you want now better, faster, cheaper. And I always say that doesn’t exist. You can one or two of the three, not all three; that’s not possible.  If you get that promise, somebody is lying because you’re going to be stuck.  

If you want faster and cheaper, you’re not going to get better because you have to reduce the scope.  You have to cut corners somewhere because things cost. And for someone who has really done research, they realize how much work goes into it and why is it worth to pay for it, right?  And so, when you have a client that doesn’t really, has never done it, that’s the hardest sale and the more difficult relationship because they don’t get it. And so, we work directly with end-clients; we work with other research agencies as partners; and also, we work with advertising and marketing agencies when they need.  We are certified as women-owned and minority-owned, and so sometimes that helps to win a project because companies need supply diversity too. And so, that’s part of the three segments that we work on.        


It’s really interesting.  One of things that I’ve observed is there used to be a fair amount of rigor associated with being a researcher.  And one of the other guests I had on earlier today, he said that (actually, it was Simon Chadwick) said that when he started his career in research, he couldn’t do research for two years.  It was like that kind of a mentorship.  


Yes, yes, Simon actually was my mentor.


And nowadays, it’s out of the box.  It’s like expected. “Oh, here’s the template.  Do it.”


Yeah, yes.  And the automation also has a lot to do with that perception too.  Now it’s more and more of that. Actually, Simon was my mentor when I was at the MSMR program at UTA, which is a very good program.  We get out of school very purist: we want to do it the right way. And then reality start adjusting. And still… you mean their projects were…  I have to pass because the request for doing certain things there like… “No, I don’t feel comfortable delivering that. It’s just too much violation of the principles.”  That’s the challenge. So the best clients will be the ones who really understand and value research. That’s my final answer.  


Perfect.  Michaela, did I say your name right?




Michaela, sorry.


That’s OK.


If someone wants to get in contact with you, how would they do that?


Well, they can go to  We are there. Our phone number is there. You can find us there.


That sounds like a good way to do it.  Thank you very much for being on the Happy Market Research Podcast.


Thank you.  Thank you for inviting me.


Are you going to the event tonight?  The comic thingy.


Yes, the Second City Show.


Yeah, that should be a lot of fun.


That’s one of the good reasons I came to the conference too.  


Yeah, no kidding.  Everybody, I really appreciate you taking the time to listen to this episode.  If you liked it, please do me a kindness and take time to screen capture, share it on social media.  As always, your reviews are immensely appreciated. Have a wonderful rest of your day. 

NEXT 2019 Podcast Series

NEXT 2019 Conference Series – Lesley Rohrbaugh – Consumer Technology Association

Welcome to the 2019 NEXT Conference Series. Recorded live in Chicago, this series is bringing interviews straight to you from exhibitors and speakers at this year’s event. In this interview, host Jamin Brazil interviews Lesley Rohrbaugh, director or research at Consumer Technology Association.

Find Lesley Online:




Lesley with CES.  Thanks for being on the Happy Market Research Podcast today.  


Thanks for having me.


We are live at the NEXT Conference in Chicago.  What do you think about the show?


It’s amazing.  It’s so nice to be here with so many of our colleagues in the research community.  A lot of great turnout from all sides of the research community. So we have exhibitors here, and we have analysts on the corporate side, the non-profit side.  So it’s great to pick each other’s brains.


CES is a big deal.  I mean it’s the trade show of trade shows of trade shows, right, the mother ship.  But I hear that you guys have a track specific for research. 


We do.  So, a few years ago, we started a research summit at the show that precedes the actual show days.  And we have a ton of participants there. We have multiple panels; we have speakers from big and small companies.  So we have startups and then we have the big, large corporations. But it’s attended by a few thousand people, and all we talk about for a few days is just research, research, research when it comes to technology.      


So, the application of technology to enable customer conversations sort of bent?  Is it user experience research, market research, kind of broad umbrella of just consumer points of view?  


It’s a mix.  We also have B-to-B in there as well.  So, with technology, you have to be able to adopt it somehow, and what better way than to talk to companies in their space and see how they’re actually using things like voice, things like AI, robotics, all these different areas.  So it’s a mix of folks, and it’s really interesting to see the application of things like Artificial Intelligence being built into the data computerization and seeing how people are actually making these ideas come to life.    


I loved your talk this morning, especially as it related to smart devices.  The microwave is still frustrating to me because of all the stupid buttons on it.  




And a microwave that actually can identify the objects, the items that are going in there, and then smartly uses that information to not burn the popcorn, as you aptly said, right?  Why don’t you give them sort of the highlights of your 30-minute chat? 


Yeah, sure.  So, obviously, 30 minutes is not long to talk about the entire 350+ billion-dollar technology world.  But some of the highlights include 5G: where we are? where we’re going? what it means? who’s driving it? which is really the industry as opposed to the consumer, which is a little bit different.  Additionally, we talked about Virtual Reality, Augmented Reality: how that impacts both the consumer and research and insights. And then also things like healthcare technology. We touched on that a little bit.  There’s a lot going on in that space. But, overall, I think some of the underlying technologies that we see throughout all of these products include Artificial Intelligence and the real state of Artificial Intelligence and where we are.  Things are not necessarily just smart anymore; they’re intelligent. So, I think that’s an underlying theme we see in a lot of these products.   


That’s an interesting distinction that I have, probably moronically, never made before your talk.  This idea between smart and intelligent. Because one is more of information in action – it’s doing stuff with it and then forms and outcomes.  Going back to the smart or intelligent whatever, microwave, that’s actually really interesting how you’re seeing a continual evolution in this new, uh, with AI. 


Yeah, that’s right, and it has so many different applications, whether it’s synthesizing these massive data sets.  I know I brought up the example of using AI for segmentation analysis and how much time it would save you. But also, just being able to read people’s emotions in an intelligent manner.  So, that facial recognition, that object recognition, that’s where it kind of comes into play. You know what are people’s emotions really telling you in a real-time fashion?  


If somebody wants to learn more…  The audience for this particular podcast is centric to insight professionals, UX professionals, and market research professionals.  If they want to learn more about what CES is doing in research, how would they find out that information.  


Sure.  So, CES is actually produced by the Consumer Technology Association, who conducts the research.  We can be found at  And we have an entire research page, and we typically publish about three dozen studies a year, and that’s everything from B-to-B research, to B-to-C, but also it covers a ton of different topics and everything from your traditional tech like television and video and audio all the way through emerging tech studies such Artificial Intelligence. 


Love it.  Lesley, thanks for being on the Happy Market Research Podcast today.  


Alright.  Thanks for having me.


Of all the guests I’ve ever solicited to be on it, you were the easiest.  Like you literally just like pivoted, walked right to the mic. It was hilarious.  


Well, good to hear.


Thank you so much.  And thank you very much, Insights Association, for hosting us at the event.  Really appreciate it. If you found value in this episode, please take the time to screen capture, share it on social media.  We would love it. Reviews are always appreciated. Have a wonderful rest of your day. Bye, bye. 

NEXT 2019 Podcast Series

NEXT 2019 Conference Series – John Tansey – Dapresy

Welcome to the 2019 NEXT Conference Series. Recorded live in Chicago, this series is bringing interviews straight to you from exhibitors and speakers at this year’s event. In this interview, host Jamin Brazil interviews John Tansey, director of business development at Dapresy.

Find John Online:




My guest today is John Tansey with Dapresy.  I still can’t say the name.


Dapresy, Data Presentation Systems.


Dapresy.  I don’t know.  Anyway, I apologize.  


That’s OK.


Rudy has been on the show before.  I’ve been a big fan of your guys’ brand.  What is going on right now in the world of Dapresy?


I’m still trying to figure this out because this is my second week on the job.  




Thank you very much.  It’s exciting and, overall, joining, it’s clear to me they’re experiencing a lot of growth.  There’s a lot of hiring. We’re opening up a new office in Sarajevo.


Congratulations.  That’s huge.


A lot of it is sales and marketing.  And then we’re trying to figure out who’s going to do the work that we’re selling to.  So, there’s positions open for that as well.  


How exciting.  I mean you guys offer industry-leading, customized data dashboards that pull data in.  I always thought of it as quite literally as a BI for insights. Is business evolved into a different direction or is that still the core competency?    


It’s still the core.  And the target has always been mid to large market research companies as well as enterprise but we now have a Do-It-Yourself option coming out, which is a much lower entry point in terms of cost.  So we’re going back to many of the people who maybe have been turned off in the past with this new offering going forward.  


So that is going to unlock a whole different set of customers, it sounds like.




What do they look like?  What are the personas? Do you know?


Users of?  These are people who respond…  Well, the short answer is, “No, I don’t know.  I’m trying to figure this out.”  


I love it.


You have owners buying it.  You know from smaller market research companies and analysts using it.  


You think about like enterprise research or enterprise solutions used to be sold at an executive level.  These are like big bundled products, expensive, difficult to deploy. And now, like in modern context, it feels to me that any business that’s here as a technology-based business that doesn’t have a premium DIY (I don’t need to talk to anybody) offering in the next three years is going to have a big problem with growth.  You have to operate at that level. One of the things, I think, that SurveyMonkey has done a great job of over its whatever 15 years is exactly operating in that space. Like it fits like a CRM or SalesForce or MailChimp. I feels like a tool that I can just plug into my platform so that I can easily get to things. But, on the other side of it, I really believe you have to have the Do-It-For-Me option as well because, especially in insights, things can get complex; people can get busy.  And we’re happy to write checks to solve those problems.      


Sure, I think that works well for enterprise and people who, unlike you and I, did not grow up in this industry and did not work as a research analyst and kind of just need a reporting solution and somebody to do it for them.     




And the thing I love about the product is it’s designed for market researchers.  You mentioned Business Intelligence tools. You wrestle when you try and squeeze market research projects into those tools, but this is designed for people like us.


Exactly, exactly.  All right, great. If somebody wants to get in contact with you, how would they do that?


Well, I don’t have any business cards.  They can dial 603-978-6594.


Well, and do you have an email address?  


I do.  John J-O-H-N.T-A-N-S-E-Y@D-A-P-R-E-S-Y – Dapresy.


Got it, and we’ll include that information in the show notes as well.  John, thanks so much for being on the show.  


Thank you, Jamin.


Everybody else that’s listening, appreciate your time and attention.  Special thanks to the Insights Association for hosting us here. Have a great rest of your day.

NEXT 2019 Podcast Series

NEXT 2019 Conference Series – Ellen Kolstø – IBM Q

Welcome to the 2019 NEXT Conference Series. Recorded live in Chicago, this series is bringing interviews straight to you from exhibitors and speakers at this year’s event. In this interview, host Jamin Brazil interviews Ellen Kolstø, Design Principal at IBM Q. 

Find Ellen Online:




My guest today is Ellen.  Ellen’s been on the show before.  She’s with IBM Q, right?


Well, yeah.  I’m actually now working in the quantum computing space.  I used to work on AI for two-and-a-half years, which is how I happen to be here, talking about AI.  


How did your talk go?  


It was great.  We had a lot of really interesting people in the room, a lot of discussion around “Are you ready for AI?”, “What does it involve?” in terms of having successful AI or AI that works reliably.  So we had a great discussion. 


That’s awesome.  This is my first time at the NEXT Conference.  Have you been here before?


No, this is my first time too.


What do you think?


Oh, I think it’s great.  I think that anytime you can get a bunch of market researchers together to talk about what’s going on, it’s super helpful.  And you learn something from what everyone else is doing. I took a bunch of mental notes even in my own session of people mentioning things.  So, yeah, it’s helpful.


It’s fun; that’s fun.  So, what are you seeing as…?  Were you able to participate in any of the earlier sessions?


[Sighs] No, I flew in late last night.


That’s what I thought.


And then we were preparing for this morning’s presentation.  So, I learned a lot in my own session so far.


Ahh, that’s…  What’s your big takeaway?


That everyone is super excited about AI, but they don’t know how to get started.  




And my view point on it is it’s the same sort of test-and-learn scenario of any part of market research:  take a small part of what you’re doing, test it out, figure out what you’re learning and move from there rather than trying to boil the ocean.  And I think that was what a lot of the people who tried it were talking about as well in the session.  


So, what is one practical way…  Can people engage with you or IBM to help in a similar way to like AWS?


Uh, well, that’s a good question.  Uh, that’s a hard one for me to answer.


Are engagements usually…  Like when I think about IBM, you know this is enterprise-to-enterprise sort of like an engagement, right?  And that’s kind of like the sweet spot, I think, of why people turn to IBM in that kind of context.


Right.  I mean we have cloud services and we do have Watson on the cloud.  So you can..


Oh Watson, yeah, right.  Exactly.


Yeah, you can access Software as a Service through IBM as well.  So you can engage in that way. I just wasn’t sure if you meant me personally, or…


No, no, no, I’m sorry.  That was a bad question.    


I was like, “Well, you could, but that would be a LOT of people.”  


That would be a lot of people.


Uh, but you can actually use IBM’s products, but we create products in the space that help other people build models.  So the question is whether you want to build your own model or whether you want use a service that already has models built for you. 


Oh, got it.  That’s actually super interesting.


Yeah, it’s a decision to make for sure.


Good.  Yeah, so, you’re flying out today?


I am.


Are you really?


I’ve got client meetings tomorrow.  So, it’s like no rest for the weary.


No rest for the wicked.  I think is how that goes.  


I know the wicked.  I like to say “weary” instead of “wicked,” but yeah.


My grandmother taught me that.  Anyway…


I know.  My grandmother probably taught me that too.  


That’s hilarious.  Ellen, thank you so much for being on the Happy Market Research Podcast.


Oh, my pleasure.


Safe travels to you.


Thank you.


Everybody else, please take the time to rate this show.  Your reviews allow other people like you to be able to find it.  If you’re not attending the NEXT Conference right now in Chicago, I’d encourage you to check out the Converge Conference that’s coming up in fall; actually, I think it’s December.  So it’s wintertime; it’s the last conference of the year. It’s a fantastic one based out of L.A. Out of 2018, it was the best conference that I went to. No offense to any of the other GreenBook.  It’s really small, super intimate, fantastic content that we covered there. Anyway, have a wonderful rest of your day.  

NEXT 2019 Podcast Series

NEXT 2019 Conference Series – David Paull – Engagious

Welcome to the 2019 NEXT Conference Series. Recorded live in Chicago, this series is bringing interviews straight to you from exhibitors and speakers at this year’s event. In this interview, host Jamin Brazil interviews David Paull, CEO of Engagious.

Find David Online:




David Paull, Engagious.


Yes, sir.  Great to see you.


One of the leading podcasts in the market research space.  What do you think of all the lists for podcasts that’s been popping up?  


Well, I think it’s great that the industry is taking notice and wanting to share it.  I’m certainly gratified by the ones that we make it on. You know you sit in…  


Which is all of them.


Well, thank you.  Likewise, for you.  I’m always in good company.  You know you sit in a room with a microphone and you talk into it.  And you hope that on the other end somewhere, someone is going to listen.  And to this day, podcast analytics are still not great.  


They’re terrible.


So it’s hard to know if people are listening.  It’s even harder to know how long they’re listening.  I don’t care if someone starts it. I want to know if they get past 30 seconds.  Are they actually sticking with it so that I know there’s value in the work we’re doing?  


On your platform, are you able to deduce if they actually listen at all?  All I’m getting is downloads.  


Yeah, it’s really just download.  We also put them on YouTube, and YouTube will tell us if we get a view.  But they count a view after 30 seconds, which doesn’t do me..


Yeah, totally.  How’s your YouTube?  So, this is one of things that actually I’m doing right now.  It started last week. We’re taking all of our backlog of audio, and we’re then converting it into YouTube.  Are you doing that?


Yeah, we’ve done that from Day 1.  We create a video version, and for most of them, when I’m not recording video, it’s just a head shot over a… 


Something moving.


Yeah, something moving.  But we do get… YouTube is the No.2 search engine in the world.  So it only makes sense to have your content there. And we are starting to do more and more video interviews, either in person or over Zoom or something.  So those fit really nicely on YouTube as well.  


That’s awesome.  So, you are using Zoom.  I’m actually thinking about converting over to Zoom for the interviews as opposed to…  I have a special like fancy-pants app that gets rid of latency issues, but it doesn’t do video.  


Right, right.  And so, Zoom, I will tell you, nothing’s perfect, right?  So, Zoom is a great application. The video quality is pretty good.  Sometimes the audio will actually be slightly out of syn with the video.  There’s latency on the audio. Now, the fix for that, of course, is when I bring it into final cut to edit it, I just break off the audio file and I move it a few half-seconds one way or the other and everything synches back up.  So there’s work-arounds. It’s a little bit of a hassle, but…   


But the good news is that it doesn’t sound like it’s an elongated… 


It doesn’t get progressively worse, yeah.  It seems to just be off a little bit.  


Favorite guest so far this year.  Who is it?  


Favorite guest so far this year.   


We love them all, we love them all.


It’s an episode I haven’t even released yet.  He’s a graphic designer out of Portland named Aaron Draplin.  He’s done work for Nike. He’s a prominent speaker in the graphic design world as well.  He’s a larger-than-life character. He told me some incredibly funny stories as well as some really useful tips.  We’re probably not going to release that until later in the year. So, that’s a little anti-climactic. I got to talk to Eric Solomon, who was a keynote speaker here.  Having worked at Spotify, and Instagram, and YouTube, and Google, he knows just a little bit about technology. So he was fascinating too.     


Yeah, that’s not a bad guess.  I’m definitely going to be tuning in to that.  When’s that one up?


That one’s up.  Yeah, that one’s up.  We did that one a couple of weeks ago in promotion for this event.  So, yeah, he’s here to today. We’ll have to make sure he stops by and chats with you.  He’s great.


I tell you what.  One of the things I’m really in love with is the cross-support that’s happening inside of the podcast community.  Going back to YouTube really quick, being one of the largest search engines, one of the hacks that… And I think this is something that would be really good to explain to people for discoverability.  You think about the modern buyer of whatever it is that we’re selling is using the internet oftentimes. That plays a part in the consumer journey, right? So the better your SEO… Are you seeing YouTube as an overall improvement to SEO?  


Yes, because if you post your YouTube videos properly and you tag them properly…  YouTube’s part of Google. It’s not that people searching on YouTube is the No. 2 search engine.  It’s that when people are searching on Google, YouTube results are getting served up often at the very top of that results page.  So, if you search on the right terms and I’ve tagged it right and I’ve built up just enough viewers, my video interview might show up above many of the first-page search results.  


Thanks pretty awesome.


So, it’s trying to hack Google a little bit by just using a whole other channel of theirs.  


And the best part about it is it’s actually giving consumers value at a click, as opposed to sales content, which is very, very powerful, I think, and one of things that Goggle’s seeking to prioritize as well.     


Yeah, absolutely, we don’t sell anything on the podcast.  It’s all about delivering value, delivering learnings, trying to give back into our community.  And, of course, the by-product there is, if people are interested in what I’m talking about and they want to seek me out, then all the better.      


Awesome.  So, NEXT conference.  What do you think?


I like this conference a lot.  I really love the intersection of research and technology.  It’s not super huge; so, you really get to talk to everybody.  I really like it. I like everything the Insights Association does.  We’re a supporter of theirs, and I think they put on really good quality, targeted events.  So I’m happy with this one.


Awesome, man.  Hey, thanks so much for being on the Happy Market Research Podcast.


Thanks for having me, as always.


Everybody else, we will link David Paull’s information in the show notes.  Please, please, please reach out to me if you have any questions about podcasting, or what in the world Engagious is doing these days.  Have a great rest of your day. 

NEXT 2019 Podcast Series

NEXT 2019 Conference Series – David Almy – Insights Association

Welcome to the 2019 NEXT Conference Series. Recorded live in Chicago, this series is bringing interviews straight to you from exhibitors and speakers at this year’s event. In this interview, host Jamin Brazil interviews David Almy, CEO of Insights Association.

Find David Online:




Hi, this is Jamin.  You’re listening to the Happy Market Research Podcast.  David Almy is the guest today. I am going to play our conversation with you.  We kind of picked up at a neat spot, I think, talking about trends in the space, how insights are becoming a cornerstone of success for modern businesses.  Enjoy.


Well, from ’82 to ’84, I wrote an article called Business Aviation – The Fortune 500, which was a correlation of the use of business aircraft to financial performance.  


That’s fascinating.  


Yeah, and so, and it looked at the Fortune 500 and said that if you owned a business jet, for instance, how’d you do.  And I did it for three years times 500 companies, and big surprise to you probably is that there was a correlation between performance and business-jet ownership. 




Now, the great question then is whether or not it was causal or…


Always the question


…or otherwise.  Having done that 82’ to 84’, very long time ago, you come here and you listen to your commen


Throw that thing out there.


And I have never seen any of that correlation study in this industry.


Why is that?


Because this industry is not as evolved, not as mature (a funny word to use) as the business-jet industry, which has been around for 60, 70 years bigger 




Yeah, yeah.  So it’s an evolution of the industry’s group-think, if you will.  So, you announced to me this morning Watermark. And I’m going like, “Holy crap!”  This is exactly what I need, been looking for and did 38 years ago or whatever it is.




And I went to look for it and couldn’t find it.  So you’ve got to tell me where do find it.


I’ll give it to you.  I have it on my desktop, well, laptop.


OK, good.  It’s a keen interest, and I think particularly when you looked at the Insights Association, our mandate is economic development, is what it is.  It’s a not-for-profit 501C6.


That’s interesting you say that.  I did not know that.  


Yeah, that’s what we’re here to do.  Per the IRS, that’s our reason for being, is the growth of the industry.  And it’s a lovely term in the IRS regs, but our mandate is the growth of the industry from we draw our members.  That’s different than the growth of our members because it’s broader, bigger than that. And I’ve always loved that turn of phrase, you know.  So, come back to your Watermark, I’d really love to see that study and probably, if they want to do it again, participate in it somehow, support it ‘cause that’s… 


Yeah, I’ve just been in Twitter conversations with one of their principals.  I can’t remember her name offhand but, yeah, I’ll do whatever I can to help connect.  


Yeah, that’d be great, that’d be great.  


For sure.  I mean it’s a big deal, and it’s a great paper, I think.


It is a big deal, and it’s one that I kind of know in a different life.


Yeah, totally, it’s funny that we haven’t been driving that conversation though, to your point.  And I think that that’s systemic, or it’s endemic – anyways, it’s one of the “demics” – where we have had this like stuck-in-time framework.  And, if you pull back, you can see that… I started my research career in ’96. Market research was its thing in a department inside of a large corporation.  And then, tools came around. Remember Confirmit launched one of the first VoC at scale using Google ad words. And so, VoC all of a sudden became this like different thing.  NPS then became a component. Neither of those now necessarily sit underneath market research: they sit underneath marketing. And then you see user experience that is now also been birthed out of market research for all intents and purposes; they just don’t know it, so like an abandoned child in every way.  And so, what’s happening is on the vend diagram, they’re doing the same or similar type of work as market researchers but they’re all young. I haven’t seen many UXers (There’s a few) but many UX researchers that are like us, age-wise. They’re having to figure it out like they’re starting from nothing, which is really funny because basic questionnaire design and all these kinds of things that we’ve had in our rubric over time that to know that something’s good or bad in terms of research, they’re trying to define the rubric.  And then, you look at LinkedIn #marketresearch, 350,000 people follow it; #userexperience over 4 million. So, you see what I mean? So now, it’s like, “Gosh, we really have a stage that we should stand on and assert ourselves because they’re looking for the parental role. I really believe that.         


Right, right.  Well, the question that you’ve drawn is what’s the identity of the community and why is it segmented, fractured, spinning off asteroids in seemingly not cohesive.  I asked Simon Chadwick one day this year… I said, “When was the last time that the industry was really cohesive? Everybody was on the same page; everybody was doing the same thing?”  And he instantly said, “2004.”      




And I said, “OK, what was going on in 2004?”  And he kind of made the case that we knew all about secondary data in 2004 but it wasn’t here yet.  So, everybody was doing primary research and custom and it was all, it was like copy-cat around the room.  And then this cliff came, and the world started to just radically change. They knew it was coming; it wasn’t a surprise. It was just the ball started rolling down the hill and picking up moss in different directions and going in different directions.  I also think, talking about the culture of the industry, that we tend to self-fragment almost. Instead of kind of rallying around a center point, we tend to say, “Oh, well, we’re special; we’re different; it’s what we do.” As I say sometimes (it’s horrible and pejorative) but we get paid by the number of tabs that are in the binder.  So we got an economic incentive to fragment, which is bad. I don’t think we’re quite there yet. To take that home a little bit, we’ve been having a lot of conversations. I’ve been having a lot of conversations in the last year about whether we’re a support function or a leadership function.  

And it’s my view…  I think culturally, traditionally, we have been a support function.  That’s how we view ourselves: well, we do research and we come up with the ideas and we kind of slip them under a door.  It’s like your hotel bill at the end of the stay. And then there’s this “Oh, try to do this or do that.” But what I laid out yesterday, which was the first time in the U.S. that I had laid it out with the four-step process:  problem, research process, outcome or insights, and implementation. That’s the definition of a leadership function rather than the definition of a support function with implementation being key to it. This came from a conversation, kind of this concept came from a conversation I had with the Head of Function for Labatt in Toronto, of all places.  So, she’s a corporate researcher in-house at Labatt’s breweries, Alyssa Rodrigo, great 32-year-old person, by the way, there. I said, “What do you do? What’s your life like?” Corporate researcher, I’m asking the question to. And she said, “Well, 75% of my time is spent implementing the insights that we come up with.” And I said, “Really?” And I said, “So, 25% of your time is research, and 75% is implementation?”  And she said, “Yes.” And I said, “Do you know how rare you are?” And you could hear her blush over the phone; it was pretty funny.  

But, when you think about what she’s doing, she’s not alone in that, and I think there’s a trend in that direction because of realization – just as you talked this morning – realization about customer centricity and the key to the future and key to success and key to a company culture’s effectiveness.  She’s on the frontline; she’s actually doing it: 75% of her time. And I said, “How do you do that?” And she said, “I set up meetings; we have discussions.” I said, “Well, what about the research part?” She said, “Well, we outsource most of the research.” She said, “I direct it.” But she was really focused on the 75%.  And she said at the end of the day, the thing that makes her happiest is seeing changes in the results in the company’s bottom line as the result of the changes that she’s done, the implementation that she’s done. That’s, to my ear, that’s a completely permanent position that is essential to the company’s success and will never be optional because it’s central: it’s at the heart of who they are and where they’re going.  


When you think about…  We hear a lot about ROI on research, which is this very vacuous framework.  One of ways I’ve heard it implemented or measured is how many times are they cited?  How much is research being cited in the actual business cases of the next step that we’re going to take?  Like next quarter or whatever. So it’s like how much voice is really being represented of the customer. I mean that in a generic sense, a market research way, consumer insights into the subsequent business plans.  Is it all just gut/feel from the executive level? In which case there’s no ROI on the research. We’re wasting our time. So, I mean this is a one-use case; there’s others. But that’s one that I think I can really kind of like get my hands around.             


So, fun parallel for you.  I looked at (again, prior life), I looked at the uses of business jets.  And I asked the question, “So, what are people using business jets for?” And, of course, you think, “Well, the CEO got to a meeting in Oklahoma City or something or other,” right?  And that’s easy. That’s one. I spent 12 years on this topic and found about 37 different uses that I was able to identify of how companies were using it. The reason it took so long is that it was considered “secret sauce.”       


Oh, interesting.  


It’s proprietary.  It’s part of our strategy to succeed.  So we don’t broadcast it, and over the years I noticed that one in ten, only one in ten, companies that I would talk to would actually tell me real stuff and when you get to insights, the question is, “How many insights have I actually had handed to me?”  “Here’s what we’ve done at our company insights-wise. Here’s the outcome of the research process.” And the answer is, “Those are tough to come by because they’re ‘secret sauce’.” It’s the undiscovered country, I think. And I think there needs to be a lot more research on what those outcomes actually look like.  I think we’d be surprised. I don’t need to know Company X is doing this particular thing and that particular thing, but the tone and tenor of them, the flavor of them, the innovation that’s built into them – I just want to see and understand a lot of them to get my arms around what the benefit actually looks like. That’s before the implantation but just what comes out of the research process.      




There’s maybe two paragraphs that says, “Build this widget,” or “Offer that service,” or “Change this this way,” etc.  And I think we’d be fascinated and I think the community would be fascinated if we could benchmark effectively in some way to show each other what we’re doing.   


Yeah, I mean in a lot of ways I think that’s the function of Insights Association, is sort of that opportunity to come together as a family and have Christmas dinner and get to know each other again, right.  So, it’s… I think the Christmas dinner is kind of a good analogy anyways too because you have sometimes that you don’t necessarily like that uncle or whatever.  


Well, we have some honest broker qualities that we’re supposed to listen and not disclose but help.


But getting to, I think, the broader point, which is we are in a stage right now of a rising tide.  And this is an opportunity: I believe over the next three years is… will be in my career probably the last time I see anything like this happen where you’re going to see insights being asked, being pulled into the boardroom.  So, I was talking with Simon about this recently… I believe there’s going to be a function that’s created that is like Insights Oversight role from a consumer insight perspective. 


There should be.


Absolutely, because right now it’s so fractured.  Everybody from the intern to the CEO is doing research.  There needs to be some overarching governance around it not just from a compliance perspective – yes, to that – but also from a quality of control perspective and interest in the customer.


It’s an essential role.  It’s an essential specialty.  It’s not an ad-hoc thing. It’s its own locus.



[14:09] has to be there, and it has to, I think we need to evolve.  And that’s why we’ve just adopted this new tagline that talks about creating competitive advantage.  And that’s that central purpose. You got to have somebody that fundamentally works on that 60 hours a week.        


Yep, exactly.


And that’s where we are.


Super exciting.  Insights Association is doing…  What is new in the Insights Association? 


Well, we have a new code that we just finalized in April, which we’re going to mail to the membership.  We also are surveying corporate researchers right now; we got a research study in the field and a phase 2 coming with regard to the uses of research and the methodologies and tools as well.  We’ll get the implementation part of this, and I think you’re going to see some emphasis on the implementation of insights in both this year and in future years because that’s a big topic. 


Got it.  The next event for you is…


CRC, Corporate Researchers Conference, in October.  There’s a small event, which is CEO Summit in Edinburgh coming up. 


Is this the first time you’ve done international?  You did…


We did London last year, London last year in concert with MRS, and we’ll be back with MRS in Edinburgh in September.


Who’s hosting?  I know here it’s Merrill and Steve Schlesinger.


It’s Merrill and Steve, and Glen Foreman is going to be involved as well in Edinburgh.  I think it’s right after Congress, so…   




…Wednesday, Thursday, Friday right after Congress in Edinburgh.  CRC will be in October in Orlando, and hope to see you there.         


Oh, yeah, you will, for sure.  We’ll be exhibiting or something.  We’ll 100% be asking to do Happy Market Research on the premises.         


OK, good, good.


Yeah, good.  That’s exciting.  If somebody wants to join Insights Association, how would they do that?  


Website’s right there or give me a call or send me an email.


Love it.  



Perfect.  David, thank you so much for being on the Happy Market Research Podcast today.


Thank you.  


Everybody else, please take time to…  I want you to share the episodes. What I really want you to do, if you’re in the insights function, take the time, go online, check out the Insights Association.  I’m a member. I find tremendous value: they produce great content; they have a very active forum as well for you to ask questions from your peer group; look for mentors, whatever; look for vendors.  It’s a really good supportive community. Take the time. Investigate it. I’d encourage you to sign up. It’s a nominal fee relative to… It’s certainly an outsized value aspect there. So take advantage of that – Insights Association.  Have a wonderful rest of your day!  

NEXT 2019 Podcast Series

NEXT 2019 Conference Series – Brianna Sylver – Sylver Consulting

Welcome to the 2019 NEXT Conference Series. Recorded live in Chicago, this series is bringing interviews straight to you from exhibitors and speakers at this year’s event. In this interview, host Jamin Brazil interviews Brianna Sylver, president of Sylver Consulting.

Find Brianna Online:




We are live at the NEXT Conference.  Here’s an interview I was able to pull off with Brianna Sylver with Sylver Consulting.  Enjoy.


Congratulations on a great talk because at lunch I had a table of…  I was sitting at a table of six other people, me and… seven. I asked them their favorite moment of the day, and they all unanimously agreed that it was the Storytelling Workshop.    


Oh, that’s fantastic to hear.  


Yeah, swear to God.  Yeah, swear to God. Anyway, and then they talked about it for a good five to ten minutes afterwards.  It definitely hit a chord. Is that something you’re seeing in the space? Like, we need better storytelling.  


Definitely, I think we need better storytelling.  And I think… part of the reason why this talk exists is because I feel like a lot of the stuff that you hear about storytelling today is like it’s sort of very 10,000-foot surface level as far as how to do it.  And this particular framework really gets into the nitty-gritty as far as how to structure a story and what is the purpose and why you’re telling the story. Who are you telling the story to? How do you connect with them?  How do you communicate? So it goes through all the of four steps associated with that. But it makes me super happy to hear that because you know sometimes you’re talking and you’re like…    


What’s happening?


Yeah, like are people resonating with this?  Are they not resonating? So I wasn’t quite sure.  It’s pure delight right now.  


Yeah, yeah, seriously that’s absolutely the truth too.  Anyway, it was good; it was really good, really valuable.  Another thing that I think you’ll find interesting is… So, I’ve done about 140 episodes, which is to say I’ve talked a lot.  I always ask, “What is your biggest problem?” in a full interview. Inevitably – I would say 7 out of 10 times – storytelling is the big problem, like a lack of storytelling inside of the organization, which then connects… effects change or empowers the insight or extends the power of the insight to effect change.  Really, the quality of the story as opposed to the degree of quality of the research, which is a little bit counterintuitive, is in a lot of ways the missing ingredient from a brand point of view.          


I would 100% agree.  it’s interesting because I came to the market research industry by way of design.  So, I’m actually a graphic designer initially, and then I have a master’s degree in Human Centered Communication Design and Design Strategy.  And part of the value of design is sort of taking a human-centered design approach is in your ability to visualize the vision, and a big piece of that is storytelling.  And so, that’s always been just sort of my framework for how I approach my work. And so, when I started to bring in more of the market research side, which I did purposely because I think sometimes the insights gathering from a design perspective at least from a method and even frankly from a quality perspective sometimes can be a little lacking or flat…  And so, I found that the market research side of the equation really helped to create robustness that was needed truly I think to make really solid decisions. But now bringing in these two pieces of like, I would say that the market research is kind of the “What,” and the design is the possibilities for what that could mean.  


Oh, interesting.


And then the other aspect of our business brings a strategy and that’s kind of like, “This is what I’m willing to do.”   


Right, so, tell me a little bit about your business.


Sure, I’d be happy to.  So, my company is called Sylver Consulting.  We support mainly Fortune 500…


That’s Sylver.  


Sylver, Sylver Consulting.  We support mainly Fortune…


And that’s just because it’s your last name, not because gold.


Correct.  There’s a story around that too that I’ll be happy to share.  


I think it might be interesting to be honest, but we’ll get there in a minute.


So, Sylver Consulting, we support Fortune 500 organizations, mainly Fortune 1000, to really manage the transformation of their brands and offerings into the future.  So, we sit at the intersection of market research, design, and strategy. And we really support organizations to enhance their brands and figuring out why they deserve to exist in the forever-changing landscape, filter innovation pipelines, both looking at next gen as well as future offerings, totally new offerings, and then really aligning stakeholders around new visions for growth.  So looking at all of the change ‘cause a lot of the work that we do is really around transformational change, and it requires that people have to allow certain things to die, certain narratives or beliefs or values of the organization to die.   


That could be like a tectonic shift for an organization. 


Yeah, and it has been in many, many ways because in order to embrace visions for the future, they have to open up the space for that.  And so, there’s a lot of iterative, iterative conversation that has to happen, a lot of helping people to really interpret and understand and bring forth kind of these new visions together.  So, that’s that piece of our business.     


That’s really cool.  You guys are based in Chicago.


We are.  We’re based up in Evanston, so just north of Chicago.  


But your customer base sounds like it’s national, maybe even global.  


That’s correct.  So I would say most of our clients are in the United States, certainly some have global presence and we have capabilities to do research all over the globe.  I think, at this point, we’ve done research in 39 different nations across the world, multiple many times. But yeah… Ironically, not too many are in Chicago.




Yeah, you know it’s funny ‘cause I started the business and initially…  I started the business 16 years ago. And just sort of happened accidentally; like I call myself an “accidental entrepreneur.”  I ended up getting downsized out of a job; I had $200 in my bank account; and I was like… 


Mama’s got to make some money.  Now I get it.


And I basically leaned into my gift of talking and connected to people.  I started bringing… My first job actually was here in Chicago, and it was bringing customer centricity into the new product development process of a bank.  It’s the grace of God that that even happened. I don’t know how that happened today. But then like I would go to conferences and I would meet people and then that’s how I got into certain pockets.  And so then, I have a lot of clients in Dallas, Texas, because you meet them and then they travel to some place else. And so, just these pockets of networks start to happen. 


That’s really cool.  So, talk to me about who your buyer is.  Like who engages with you? At what level in the organization?


So, our buyers come from three different areas of an organization.  So, certainly customer insights or market research, and I would say typically it’s a director level or higher that are usually engaging us.  Also, innovation: people who really are in charge of looking at whole other futures or transformations for the company. And then sometimes customer experience departments as well.   


I’m just curious.  How do you… What’s your sales channel like?  It seems like those are hard people to get to network to.  It is like word of mouth or…? 


A lot of it is.  I mean a lot of it’s like…  I have a very, very rich LinkedIn following, and that is a big… 


I hope we’re connected.


Actually, I don’t know if we are connected, but we need to be


Oh, my gosh.  I’m going to look right after this.  


So, the LinkedIn has been really effective for me.


Their LinkedIn strategy is effective.  So it’s driving leads?


It’s driving leads, and it’s driving knowledge because, basically, people see, know what I’m speaking about.  We reference things that we doing out in the world from our company perspective. And then, what we’re finding is that then people…   I mean literally just this past week, I haven’t talked to this gentleman literally in ten years. And he reached out to me, and he’s like, “I have a project that I think you’d be perfect for.”  Haven’t talked to him in ten years, but he’s been following me on Linkedin for ten years. And so, now he feels like he knows me, right? And just actually last week, I got on the phone with somebody I met at IIeX a few weeks ago, right?  She was like knew all the stuff about me because she’s been following me. And I was like, “Wow, like this is amazing,” right? So there’s an element of trust that’s already built. And the good thing is because of the content that we share, there’s also an understanding of really how we can help.  So that then, the leads are qualified once they come.      


So, what frequency are you posting?  Do you have like some religion around that or just ad hoc?


So, I always think about marketing in sort of the “No matter what” essence of marketing where it doesn’t matter how busy you are, you’re going to do at least this and then sort of the other extra stuff.  So, the “no matter what” is at least one time a week.      


Got it.  And is that a post or actually prepared content?


It’s a post, and certainly I’ll post about this podcast, other things that are happening, but at least once a week, it’s something that’s a share.  


That’s super cool.  I think LinkedIn in market research is probably actually the most underutilized tool that could immediately create brand or customer value to businesses.  Look at the companies that are doing it. What’s so funny is like the big players aren’t doing it right, which makes me laugh. They’re like, “Look at our press release.”  They’re totally missing the point. What you need to do is add value to the constituents that decided to connect with you. And if you do that on a regular basis, there’s an affinity that’s built over time, which is the function of brand over time.  It’s just such a great, easy hack. You know the other thing I think is like podcasting. It adds overhead and that gets into bandwidth issues, but I’m such a believer. I’ve been amazed at how I personally get emotionally attached to podcast hosts that I listen to.  They don’t know me from Adam, but I feel like I could go to Christmas with them. They should be at my house at Christmas time. It’s like this really interesting dynamic that exists. And, again, it’s all predicated on the value that they bring to me.  


Absolutely, absolutely.  No, I’m finding the same.  Like I kind of got into being guests on podcasts in the last probably about a year now.  And I love the medium; I think it’s fantastic. And I’m kind of like, “Should I start my own.”  I don’t want to.       


Totally.  You should.  You absolutely should.


I should, I absolutely should and I will in time here, hopefully not too long down the road, but like… 


So, when you do it, framework-wise, you don’t have to do…  I would recommend a vlog. Like it’s the same work. It adds maybe a 10% tax on time, same money.  And the benefits are huge for SEO. YouTube is the No. 2 search engine globally, and Google loves YouTube, as it turns out.  It actually utilizes that search algorithm. And so, as people search for content that’s relevant to whatever you’re a subject matter expert in, and then, all of a sudden, it’ll be much more likely to pick it up if you have content in both places.   


How interesting.  So, do you just like…  How does that work logistically?  


There’s transcript uploads.  I’m going to talk about it when I go to that conference.  There’s a marketing conference that we’re putting on. I don’t even remember when it is.  It’s this year. I’ll send you the information.


Yeah, please do.  I would love to know more


It’s in conjunction with Priscilla, if you know her.


Oh, yeah, Priscilla, yeah.  For sure, yeah.       


Yeah, yeah.  She’s actually the chair of it.           


Oh, OK, great.


So, anyway..


Where is it happening this year?


I knew you were going to ask…  I can’t honestly remember any of it.


Yeah, let me know because I went to that conference maybe two years ago, and I thought it was phenomenal.


Oh, really.


I didn’t have much expectation going in because, generally speaking, market researchers suck at marketing.  And so, I had zero expectations going in, but I probably got more personally out of that conference – you know as far as how to fuel my own business – than I have at many.  And I came away from that conference, saying, “My gosh, the next time I go to one of these, my ad man, who does a lot of the support around these things, needs to come with me.”    


Yeah, I completely agree.  The added value of taking somebody else with you just so that you can absorb as much as possible is…  It’s 1 + 1 =3 scenario for what you’re able to pull out and then subsequently execute. Otherwise, you spend all of your time educating her on what you learned.  It’s like this time hack that’s huge. And people think of it as like they’re choking on a cheap hotel and a plane ticket. But I mean I tell you what: throw down the $1000 or whatever it is and call it a day.  It’s in Denver, by the way. I remember.   


Denver.  OK, I want to know more about that, yeah.  


I think it’s summertime; it might be fall.  I can’t remember. I’ll give you the information.




Alright, if somebody wants to get in contact with you, how would they do that?    


So, if you want to get in contact with me, probably the easiest thing is through LinkedIn.  You can reach out to me at Brianna Sylver. It’s “Sylver” with a “Y.” That’s S-Y-L-V-E-R and my first name is Brianna B-R-I-A-N-N-A.


Brianna, thanks so much for being on the Happy Market Research Podcast.


Thanks for having me.


And we will include all your contact information in the show notes.  Everybody, thank you so much for tuning in today. I hope you found a ton of value in this episode.  I actually did. Can’t wait to dive in more with her offline. Sorry. Have a great rest of your day.

NEXT 2019 Podcast Series

Ep. 221 – NEXT 2019 Conference Series – Anne Beall – Beall Research

Welcome to the 2019 NEXT Conference Series. Recorded live in Chicago, this series is bringing interviews straight to you from exhibitors and speakers at this year’s event. In this interview, host Jamin Brazil interviews Anne Beall, CEO of Beall Research.

Find Anne Online:




My guest today is Anne Beall, Beall Research.  She’s been on the show a couple of times. We are live today at the NEXT Conference in Chicago.  Anne, welcome to the show.


Thanks so much for having me.  So nice to see you.  


Yeah, nice to see you too.  So, I’ve never been to the NEXT Conference.  I assume you have ‘cause it’s in Chicago.  


I haven’t either.  It’s fabulous, isn’t it?  


I actually really like it.  It’s a smaller show. They have another show, Converge, in L.A.  I really think you should attend, at least as an attendee. I don’t even know if they do exhibit floor.  Last year was the first year. Like this one, it has just a… It’s chock full of great brands. So, good attendees…   


Great speakers.


Fantastic speakers!  Yeah, totally. And you’re speaking later on today.  


I am.


Tell us what you’re talking about.


I am talking about the role of emotions in purchasing.  So, we’ve actually done some really exciting research on which emotions lead to purchasing, which emotions lead to re-purchasing, and which emotions lead to brands getting recommended or not.  


The actionability of that.  So, understanding as a brand understands their consumer and what’s winning in the marketplace, it sounds like what you’re really offering is helping them connect the dots to the emotional outcome of that consumption or purchase or however you want to think about it. 


So, we are increasingly of the opinion that people don’t think their way through the marketplace; they feel their way through it.


Totally.  I do.


Yes, me too.  And the brands that we engage with are the brands that give us a positive emotional response.  And the brands we really engage with give us a really positive emotional response. Apple would be a good example of one.  


And Coke for me is like…  I like Coke. Actually, I don’t like Coke.  This is funny. I buy Coke when I’m not feeling good.  It’s such a weird assumption behavior. 


It makes you feel better though, doesn’t it?  Comforts you. And Coke is, actually, a good example.  That’s actually a brand we reference in my talk in terms of a brand that’s actually driven a lot of emotional connection with consumers.  And if they had access to my model, which I think they must have at some point done that… 


Hint, hint, those who are listening from Coke.


They have used those principles that we’ve uncovered extremely effectively.  


Got it.  That’s really cool stuff.  


Yeah, it’s really great.


You’re also exhibiting at…  Does Beall Research exhibit a lot?   


We, actually, have never exhibited.  This is our very first time. So we have brought the crew out.  We are, as you know, located in Chicago. So it’s a little bit easier to have my staff here and fabulous colleagues.  And we’re also showing our books that we have written. We have a new book called Reading the Hidden Communications around You, a guide to reading the non-verbal communications of your colleagues and customers.  Body language, so to speak, and other books. As you all know, we do publish a fair bit in the firm.   


Awesome.  Yeah, you guys, about two a year is what I’m seeing as the average for a decade.  


Not quite that many, but.


Not quite?


…we’re glad that you think so.  


The last book was really interesting.  I read that. It actually changed my…  So, we have little kids, a two- and a three-year-old as well as a 12-year-old daughter and some older kids, you know, second family thing.  The Cinderella, what was the title?


Cinderella Didn’t Live Happily Ever After: The Hidden Messages in Fairy Tales.


Right, and one of the interesting about that is how in fairy tales, women are oftentimes cast as the bad person, right, the enemy.  


They’re cast as very helpless or, if they’re powerful, they’re often cast as quite evil.  So, men are cast as being powerful and good, whereas women who are powerful in fairy tales (aka witches and stepmothers) are generally evil.  In fact, there’s not one nice stepmother in a fairy tale at this point. And so, that’s kind of a sad thing for stepmothers who, as your lovely wife knows, it’s a stereotype that’s just not accurate.    


Yeah, that’s right, actually.  That’s a really good point. Anyway, I loved that book.  I was super informative for me.  


Yeah, there are little girls who occasionally call me.  It’s actually not a child’s book, but it does tend to be something that young women like.  And I’ve had a young reader as young as (I think she was 12-years-old) … She said, “This has totally changed how I think about fairy tales, and I don’t want to be Cinderella when I grow up.”  


I gave the book to another friend of mine, and he and his wife actually stopped literally (I’m not kidding.  This is a true story) … After they read it, they literally stopped reading fairy tales to their kids. 




Yeah, I thought that was kind of.. wow!  Oh, my gosh! I didn’t mean… I don’t know but anyway.  Definitely, it’s very actionable.


Yeah, it’s interesting.  I’m not against fairy tales per se, but I think we want to be thoughtful about what the messages are that they do communicate.  


Well, his wife is actually very cognizant of that already.  And so, as soon as she had this, it was like, “Oh, my gosh, we got to…”


Really think about it, yeah.


Yeah, be a little more proactive in how we’re managing the bedtime content, but anyway…  If somebody wants to get in contact with you to talk about (from Coke, specifically) to talk about the work that you’re doing in emotion..


Sure, they can contact through the website.  It’s BeallResearch. com. That’s 

B-E-A-L-LResearch (one word).com.  And I’m at  So, they’re welcome to get in touch.  And I’m always, by the way, happy to talk people about our work in emotions.  And, if you aren’t at this conference, I’d be happy to chat with you about some of what we found.


Yeah, fantastic.  I’m sure there are a lot of people will be beating down the door to that effect actually because it’s such an important…  There’s three different talks at the NEXT Conference about voice. And when you get into an invisible purchase journey, which in the next three years it’s about $80 billion is going to be in voice…  So, if you’re not connecting in an emotional level, you’re really in bad shape.    


I think so.


Especially if you’re in CPG.


Sure, because the reality is that we can make choices all the time; there are plenty of products that we can buy, and we are going to be drawn to the things that make us feel good and that make us feel good, particularly about ourselves. 


And there’s no interrupt opportunity, right?  So, I can go on Amazon right now, and Scotts paper towel can connect with me if I type in “paper towels,” but an invisible purchaser journey, they don’t have that opportunity.


And why do you put Scotts in in the first place?  Did your mom use it? Do you have a sense that it’s very high quality?  And do you have a feeling from it when you look at packaging that makes you say, “Yeah, I’ll just buy this”?

Super interesting.


Yeah, it is. 


Alright, Anne, thanks for being on the show.


Thanks so much for having me.


Thank you, NEXT Conference, for hosting us here.  Happy Market Research Podcast having the opportunity to speak with the speakers here is a big honor of mine.  And I hope you found a lot of value. As always, you can find Anne’s information in the show notes. And, if you liked this podcast, please take the time:  screen capture the episode, post it on your social media. Thank you so much for all the support. Have a great rest of your day. 

NEXT 2019 Podcast Series

NEXT 2019 Pre-Conference Series – Stuart Crane & Paul Cornwell – Voice Metrics

The 2019 NEXT pre-conference series is giving listeners an inside look into companies such as IBM, Voice Metrics, Ipsos, and Pulse Labs.. Join insight leaders on June 13 – 14 in Chicago for NEXT, where you can discover how technology and innovation are changing the market research industry. In this episode, Jamin Brazil interviews Stuart Crane, Founder and CEO of Voice Metrics; and Paul Cornwell, CTO of Voice Metrics.

Find Stuart and Paul Online:

Stuart’s LinkedIn

Paul’s Linkedin



Hi, I’m Jamin Brazil, and you’re listening to the Happy Market Research podcast. This is a special episode that’s connected to the upcoming Insights Association’s NEXT conference. It is located in Chicago on June 13th and 14th. I do a lot of these conferences both inside and adjacent to the market research industry. I think this particular NEXT conference is a must attend if you’re interested in learning about what’s coming up “next”. Maybe that’s how they came up with the name. My guests today are Stuart Crane, the founder and CEO of Voice Metrics, which helps companies leverage voice, as well as Paul Cornwell. Did I say your last name right, Paul?


Yeah, you got it.


Voice metrics CTO. Guys, thanks very much for joining me on the Happy Market Research podcast today!


–Glad to be here.

–Yes, thanks for having us, Jamin!


You guys are speaking at the NEXT conference on how to integrate voice into the total customer experience. I’m really curious, given your backgrounds, when did you first recognize the voice was important?


Voice, I’ve been interested in for quite some time back in the day when I would listen to cassettes in the car and CDs in the car. I was really interesting in voice recognition:  recognizing voice with Dragon dictate, and that sort of thing. But when I realized it was really going to be big is actually when I got an Amazon Echo, I think was for Christmas in 2015, I believe, and just being able to talk to this cylinder, and have it talk back to you and start songs, and you could still talk to it while music is playing. And obviously Siri was out there. But now, it’s basically an ambient voice conversation. It just blew my mind! And then I found out that you can actually write software for it. You can write programs for the Amazon Echo. Back then, it was just called Echo. Now it’s obviously “Alexa”, and it’s a big ecosystem and everything. So I just really recognized that being able to talk to devices and have the full features of computers behind it really is going to transform things. Not that it’s going to take away the capabilities of mobile or anything like that, but supplement it in such a great way. I started looking at ways that we could program voice and got involved very early in the Alexa’s software development ecosystem and just took it from there.


All right, great. So Paul?


Yeah. So I came from an AI machine learning background prior to getting into voice, and that was sort of my segue-way into voice and where the interest came from. So actually, before I met Stuart, I was pretty hot and heavy for Alexa and the idea of building these interactive experiences. So I was looking a lot at Lex and Alexa on the Amazon side, and it just seems like a natural segue-way coming from that AI background and thinking about how these devices and experiences can be more conversational and just the technology caught up to where my head was. With the opportunity with Stuart, who had this vision, at the very beginning of what we’ve built, everything just seemed to align.


So I’m going to go ahead and share, and I apologize, I don’t mean to hijack the point, but for me I recognize it was really important with my daughter and the iPhone I got her when she was 11 years old. We were driving in the car for a three hour drive, so I just started making small talk. And I talked to her about her best friends, and her top three surprised me. It was Siri. So I wasn’t sure if she was making a joke. But we dived into that. And she goes: “Oh, you know, Siri, she’s always there. She’s talking to me.” In the context of an 11-year-old world perception, she really did not understand this concept of AI or bot. For her, it’s a voice that’s got a name, and is communicating to her. Sometimes it doesn’t make any sense. In fact, maybe that’s a lot of the times, especially in the early days, but now you can fast forward with where we are. I also have younger kids, a 2- and a 3-year old, and one of their favorite things to do is interact with Alexa, playing the hide and seek game. I don’t know if you guys have done that or not.





It’s just this construct where you can’t have a tangible game or UX –we are thinking about what that looks like, but in a voice context. For me, as I fast-forward to two or three years from now, I don’t exactly know how voice is going to look like but it feels like the opportunities for us are significant.


Yeah, absolutely. We were out in San Francisco, speaking a couple of weeks ago, and what we noticed just walking around the streets of San Francisco is half the people, probably more than half, maybe 80% of the people have their AirPods or headphones on. So once those have the capabilities, the voice assistants, built right into them, which they’re starting to do, obviously Siri is built in the AirPods, it’s going to be huge. It is just all over. It is everywhere.


So you’ve worked with a couple of market research agencies on voice surveys. What do you see as really exciting in that space? And what do you see as a material challenge at this?


Yes, that’s a good question.

The companies that we are starting to work with is in a really very exploratory way, which I’m sure we’re going to find out at the NEXT conferences. People want to see how can we utilize the voice’s system, whether it’s Alexa, Google Assistant, and Siri at some point to get data, get information, get feedback, get surveys and take them. So the agencies we’re working on right now are taking our survey platform, which is called Survey Line, and they’re basically building surveys similar in the way you would build a SurveyMonkey survey in the web app, and they’re showing them to their clients that are maybe big consumer product manufacturers or just product companies that have panels of testers out there. And they’re basically helping them to say: “Some of the things you’re doing to collect data and do market research in consumer research can now be potentially done by voice.” So they’re looking at things where they may have people coming into the homes and doing surveys by hand. And they want to lower the cost of doing a survey and consider also the convenience factor for the panelist. One thing we’re finding right now is the agency is looking at doing very interactive surveys that have a real voice behind it. So you actually have a voice actor, a voice talking the person through a product. “Pick up the product”, “Hold it in your hand”, “How does it feel in your hand?” And it’s basically through the voice assistant that way. They’re building some of the longer interactions. Some of the challenges we are working on right now is just the cadence, the pausing and stages because sometimes you might want to pause and say: “Well, you do this for a little while, and then come back and tell us what you thought about that.” And so those things aren’t as intuitive on a voice assistant because it wants to work just back and forth, back and forth. We have got some things that we are modifying to make it work in an environment where essentially the market research agencies want a hands-free experience. They do not want to have the person go to a phone or go to a laptop or any kind of tactile interface at all. They want it hands-free. And that’s what’s perfect for surveys by voice or a voice survey. But in some situations when they do something with a product and then they come back and say that. So some of the challenges, like I said, are related to cadence, pauses and delays and just getting that interaction as natural as possible, knowing that you’re still dealing with essentially a computer. As you know, IVR has been around since the 80s. So we’re taking what that had done, and saying: “Hey, this could be done on a voice assistant” and be done even better because you have full programming capabilities, you have really voice behind it, and so forth.


Stuart, I want to get to an example, if you guys have one, of a voice based survey. But before we do, Paul, I have a question in context of AI. It’s a term that we have heard a lot in market research over the last years. And the actual nailing it down in terms of how it applies and improves an outcome has been a little bit squishy in our space. Can you talk to us a little bit how the role of AI in a voice context?


Yes, absolutely. So I think, out of the box, Alexa and Google Assistant do a lot of things very well. A lot of the reasons that it’s improving over time are due to the machine learning and artificial intelligence that Amazon and Google are leveraging themselves. But we have found that there is still a gap. What we have tried to build, and I think what is successful, and what developers of voice solutions are doing, is they’re building their own sort of contextual AI. Using surveys as an example, we have actually created sort of our secret sauce to make the survey experience much smoother for the user because out of the box you run into a lot of things with Alexa skills and Google actions, where she doesn’t understand exactly what you’re trying to do. And if what you say or what she heard doesn’t match exactly what’s been predefined in those voice solutions, those skills and actions, then it can fall down. So coming at it from a pure voice developer standpoint, to me artificial intelligence, which can be a buzzy sort of word –we hear that term all the time – it just means basically having a layer of algorithms and logic that can make sense of what the user is actually trying to do with the intended action is and giving them that results. So that’s how we approach it. And I hope that answers it.


Got it. Yeah, that makes less sense. Again, going back to the simplistic example of the hide and seek game on Alexa. In that framework, it doesn’t feel 100% human, but it also doesn’t feel like going back to Stuart’s point about IVR in the 80s. It’s definitely a lot closer to the human interaction experience. All right, Stuart. Do we have an example?


Yeah, yeah, So what I’m going to show now is an example we put together at the Voice Of The Car Summit, which was a few weeks ago, that brought together people who are involved in bringing voice assistance into the car. Obviously, there’s a lot of companies involved in that, and the car companies themselves, the voice assistants, and so forth. So this example that I’m going to show is related to a survey that would be done after you go to a retailer, a restaurant or, in this case, a fast food restaurant. So everybody’s familiar with the receipt you would get at a movie theater or retailer, a restaurant, which says: “Oh, thanks for coming. Please take our survey.” And it has a long web URL that you have to type into your phone, or what have you. Who does that? Who is going to go, and take the survey, and type all that in from a printed receipt? So the idea is that we can just ask our voice assistant to take it. So the receipt would say: “Just start the McDonald survey” or what happens to be. And in this case, we did a demo for a McDonald’s fast food survey. So what I am going to do is I am going to say the wake word, and it is going to start the survey, and it would be what someone would do in the car as they are pulling out of the drive-through. So let’s give it a shot!

–Alexa, start the McDonald Survey.

–Hello, thanks for visiting McDonald’s today. We want to hear about your experience. This survey should take less than a minute, so let’s get started!

How often do you visit this McDonald’s location?

–About once a week.

–For the next few questions, rate your satisfaction on a scale of 1 to 5, where five is excellent.

Rate the speed of your food delivery.


–Rate the quality of the food.


–Rate the friendliness of the McDonald’s staff.


–Finally, yes or no. Would you recommend this McDonald’s to your friends and family? –Yes.

–That’s great. We’re happy that you’ll recommend us! Thanks again for stopping at McDonald’s and taking our survey. Next time, try our new Mushroom and Swiss signature burger.


You had to throw in the advertisement at the end, of course. But I really like this. So there’s a couple things that popped for me. I will start at the beginning. First of all, thinking about programming that survey, is that hard? I know that creating Alexa skills in general at a basic level is relatively…


-I will let Paul hit that question.

-Yeah, it’s hard to make it extremely flexible. So we built a platform to do that. I think anyone could probably build an Alexa skill or a Google action that is built around a very specific set of questions and get responses. To sort of take it to the next level, we really wanted to build something that was really self-service and something that we call a platform. But I would say the challenges were probably with supporting those different question types and collecting responses in a way that match up with what the survey creator was trying to get. So if they are looking for a rating, we have a lot of validation around. So if instead of 1 through 5, the person said “6”, we got to make sure we come back and tell that person gently: “Okay, that’s not the right answer.” And then maybe play the question again, things like that. So it’s really just having the experience of as a conversation it could be. Then, from a programming side, it was just really building the platform to support basically any type of question and answer back and forth that someone wants. And we tried to make as conversational as possible.


I do think it be really funny if you did an out take version where the correction was something like: “Hey, jackass, it is only five.”


Yes, that would be good.


That’s really feeding the point of the impact of user experience in context of feedback. You really have an opportunity to help enforce brand inside of consumer feedback nowadays. In truth, we always did. I think we are just actually starting to pay attention to it more as an industry now. But having that friendly voice is such a better experience, to your earlier example, than just having to go manually and put a URL into a web browser, which is just like filing taxes.


Yes, for sure. We think there’s multiple benefits to it. We just think it’s another way. I mean, obviously there’s other ways to take surveys. But one of the things we really like is just how we have seen some creativity with some of our customers, who are doing things like having a user do a feedback session while they are experiencing the product, which is difficult to do any other way. But voice lets you do that. So maybe while you’re trying shampoo, or whatever, and you got an Echo in the bathroom, you could actually be answering questions on how does it feel?, how does it lather?, and things like that.

We are seeing some creative stuff, and we just love that. That’s why we try to build it as just as open as possible.


In context feedback, I think it is the part that is going to be interesting for market researchers, and I don’t mean that in a narrow way but in a broad way, anybody that’s interested in consumer feedback is going to find to be tremendously valuable because the in-moment experience comment is the most valuable feedback versus the degradation of feedback because of the delay in that Q&A.




What kinds of content or insights are you actually capturing beyond the obvious answers to the questions?


It is all centered on that: the responses, as you probably are aware. So Alexa and Google don’t give anyone like raw access to the audio itself. You can’t set up a skill and then get the audio file for what exactly that user says. I want to hear their voice. It doesn’t work like that. That’s for privacy and good reasons. So they do a great job with the Speech-to-Text. So it relies heavily on Alexa and Google’s natural language processing and Speech-to-Text capabilities. So we have different question types that we support. So we support asking a user for a rating from 1 to 10 or 1 to 5, whatever they want to set up, “yes/no”, and then multiple choice, of course, and then free form, which is really wide open. So if you just want to ask for the user for some comments, things like that, we have that capability. And we have just added a new question side. We just call it “mobile phone” but it’s basically the ability to collect contact information from the user. And the way we’re implementing it right now is if the user wants to supply that, they get a text, and you need to lead to that phone, and it sort of makes that connection with the brand or whoever is conducting the survey. So we’re looking at different ways to provide value there. But as far as the actual insights, we are just looking at providing as accurate a set of data as we can per survey and then our customers will gleam the insights they want from that data.


Is the data set like a CSP file?


Not exactly. So right now that’s how it happens.


So really easy to integrate into whatever platform they’re using for their analytics. Is there any additional metadata that you are gathering, in a traditional web or web-based platform? You know, you’ve got a host of stuff like time stamps and a browser version, maybe even location?


Yes. Well, we could get what device they are using, whether it’s Google Home or Alexa on them, and then within that which type of device they get. We can only basically get whatever we are given by the platform, the voice platform, and the Google Assistant. But there is some metadata and Paul and I work with that and we provide that to some of the clients.


Got it. I’m going through the site, I literally just purchased my breakfast this morning from McDonald’s. That is maybe not an endorsement of my health, but I do like McDonald’s a lot. So I finished going through the check out. How do I get that? What’s the trigger event? Is it in cars? Is it later? How does that survey get served up to me, so to speak?


Yeah. That’s basically going to be the challenge. I think going forward, Jamin, it is basically what we call the “voice call to action”. And the “call to action” could be in so many different forms. It could be like in this example that we gave, it could be on the receipt, and it just says: “Launch the McDonald survey”, or whatever engagement or voice action they want to start. It could be printed on a product and say: “Tell us what you think. Just say to your voice assistant XYZ”, and we would obviously rebrand it to that product or to that company or whatever they want it to say. So that’s going to be going forward. That challenge is how do you implement that call to action? We are working with a company now that does direct marketing. And they have huge brands like Wells Fargo and these companies that do massive amounts of direct marketing. And they’re adding voice response into it because they get somebody who could get something in the mail and it would be “Go to our website” or “Call 1-800 number”. Well, now you just interact with us through voice, and it would launch essentially a voice interaction, which could be a survey asking them a few questions and based on how they answered those questions, it could do different things and contact them that way.

But I think it is going to be tricky because it is going to take time for companies to say:  “Where do we want to put this call to action and what should it say?” And that’s something that we can help with to a certain degree. But we’re not the experts on that so much as they are.


It seems like that’s a big partnership opportunity that you’re talking about.

I am thinking about in the market research space, we have got a host, whether it’s Dynata or others, very large market research sample providers, and I don’t know how big the industry is between $2 billion and $4 billion, if they have a voice enabled device as a variable inside of their profiles, then maybe there is that trigger event that could happen. There’s a lot of bubbles in this scenario, but assuming that there was an app that was tracking geo.


So just think of emails and how many people do SurveyMonkey or Qualtrics surveys or Zoho or whatever. Most of the time they are asking people to take their survey because they are sending out an e-mail or it’s on social media or somewhere. That could be supplemented, maybe not replaced, but supplemented with “Do you want to do it with your voice assistant? Just say launch bla-bla or “Start the XYZ survey” or whatever it is. Now, obviously, surveys by voice need to be friendlier in a voice contact so you can’t take every SurveyMonkey survey or web-based survey and just copy and paste it into a voice survey because there’s just nuances, and cadence like we talked about before, that are necessary. It’s better sometimes on a screen to sort things or see a lot of multiple choice answers, and that doesn’t lend itself to voice. But the call to action could be in similar way to the way SurveyMonkey’s and online surveys are done.


So in 2023, it’s projected that there’s going to be about $80 billion that are going to be purchased through voice. This is for me a massive number, and I’m seeing in my own user behavior that I procure or buy stuff through my Alexa device, more of the CPG type of stuff is what we’re doing. Google and Amazon of both very aggressive and gobbling up the generic brands. I know that has been well documented. So, like generic paper towels I believe is now owned by Amazon.


Yes, Amazon Basics.


So, in a voice-based, consumer journey, which is invisible, I don’t have any opportunity to intercept the consumer if I’m Scotts, for example. Why isn’t voice a bigger deal right now for the CPG spaces? Or if it is, are they just operating in secret? As a consumer, I’m just not seeing a lot of investment, and as a practitioner, a lot of research. There is not a lot of noise in the space about investment that’s being made in this invisible consumer journey.


It is a good question, and because we’re in the voice industry, we do see a lot of internal investments of companies that are building things now, but they don’t want to just rush them out to market. I was in the healthcare space for a long time, and they’re actually wanting to get voice capabilities for patients and for doctors and so forth. But the brands, like you said, they’re taking a slower approach, and they’re doing a lot of internal testing and building things. But they’re also looking at “How do we get on there?” Because Amazon and Google have basically a native interface. And as soon as you start talking, Google and Amazon know who you are. But until you open a skill or an action or some interface with that brand, they still don’t know who you are until you somehow give them permission. So it’s much more difficult beyond Amazon and Google to get that.

So that’s why we’re building in things like getting contact information right through it, and doing look-ups with codes and so forth so you could just put a code in. But there is a lot of investment going on by brands and also ad agencies. The agencies are basically thinking: “How can we get into voice?” And it’s slow for a couple of reasons, mainly because they are just trying to figure out how it all works. But also they want to be careful not to roll something out that’s half baked.


I did some analysis earlier this year on voice ratings. I was using it as a surrogate for app utilization in voice. Unfortunately, there is not a corollary. The number of rate ratings, for example, does not mean that’s the utilization of the app. But having said that, it still is really interesting to see what products are apps that are being used in a voice-based context on just frequency. And so one of the things I thought was really cool was that, I believe it was GM, that has an auto-start, voice skill to the vehicle. Again, I am intuiting. I live in California, but it’s cold outside so I’d like to start my vehicle ahead of time so it can warm up. And that’s the extent of the skill, which is very highly rated. But they were the only automobile manufacturer, even including Tesla, that had any voice-based app that was in the top 100, I think is what I pulled. So it feels like just transparency in terms of what apps are being used and by whom could be a big opportunity for whether it’s a company like yours or even a company like Nielsen for communicating to the industry what is trending from a user experience perspective.


Yes, exactly. I think it all gets back to that “voice call to action” until people know what to say to their voice assistants, what to ask of their voice assistants, and have a prompt.

It is going to take time because people know how to say: “What’s the weather like outside?” or “What’s the sports score?” They can turn their lights on and off; I have all that up in my smart home, and it’s a great way to play songs. That’s the biggest use case of all for the smart speaker. But I think once brands and companies and different entities start doing a voice call the action where they say: “Well, here’s our website. But then, if you want us by voice, say this”, you will just see that take effect. And then, it’s going to take some time, like you said: 2023, $80 billion. I think in 2023, you will just see a lot more calls to action “Hey, engage us by voice!”


Yeah, that’s right. It is the whole user journey that has to trickle down to just the knowledge of how to interact because it’s invisible. You don’t have those user prompts that you would have. I go to the Star Trek example, right, where you know you had the computer, and then there was a constant interaction with it, and then they would give commands to the computer to transfer controls to whatever. Are you seeing that as one of the maturing use cases or potential use cases where there is a voice-based Instagram feed and then the person asks to transfer that to his/her phone or something along those lines?


Not yet. It’s just kind of too much of a reach for somebody to know to do that. But I think once there is a good use case that it actually gets habitual… It’s our about habit! Turning on lights and doing things that are IoT, and if they do them often they get that habit, and then you get that. But if somebody doesn’t know to do something, to your point before, there is really no, no visual interface. In most cases, you have Echo show and you have some visual. Actually, that brings up a good point, Jamin, I do actually give it commands when I see things on my Echo show that they prompt me with an article or to do something. Right now it needs that prompting or call to action. As more companies put them out there, you’ll see more use cases, and then people won’t even need to be prompted. They will just use them. But we are still in pretty early stages on people doing things like transferring things. It will come but it’s going to take some time.


So what is one practical take-away that our listeners can gleam right now from your upcoming talk at NEXT?


I think the biggest thing is that feedback and surveys or just getting anything from a consumer or an end user, an audience is doable by voice with your own branding, and it’s just now becoming possible. So what we are going to show at the NEXT conference is basically a platform that allows you to create a survey like SurveyMonkey, but it could be branded for yourself so that it’s your own voice. It’s not the Alexa voice or Google Home voice. It’s your own voice throughout the whole thing, like I showed, and you get the data, you get to ask what you want to ask, and that the user is happy with the experience. So that’s what we’re going to show, and it’s evolving. It is still very early stages in this. But as we improve our platform, we are leveraging the capabilities that are being improved by Amazon Alexa. And then there’s also obviously Cortana and Bixby and some others, and Siri. Once Siri has the capability of program, Siri will have that as well.


If somebody wants to get in contact with you, how would they do that?


Yes, the best way is just go to our site, which is


Got it. My guests today have been Stuart Crane and Paul Cornwell of Voice Metrics. Thank you both for joining me on the Happy Market Research podcast today.


–Thanks a lot. We enjoyed it.

–Great. Thanks for having us, Jamin.


Everyone else, for more information on the Insights Association’s NEXT conference, to hear speakers like these fantastic gentleman and others, please join us in Chicago June 13th and 14th. You can also find more information on our website Have a great rest of your day, and I hope to see you there!

NEXT 2019 Podcast Series

NEXT 2019 Pre-Conference Series – Frank Kelly – Ipsos

The 2019 NEXT pre-conference series is giving listeners an inside look into companies such as IBM, Voice Metrics, Ipsos, and Pulse Labs.. Join insight leaders on June 13 – 14 in Chicago for NEXT, where you can discover how technology and innovation are changing the market research industry. In this episode, Jamin Brazil interviews Frank Kelly, Global Head of Operational Product at Ipsos.

Find Frank Online:




Hi, I’m Jamie Brazil, and you’re listening to the Happy Market Research podcast. This is a special episode connected to the Insights Association’s NEXT conference, which is being held in Chicago on June 13th and 14th.

My guest today is Frank Kelly, head of Innovation and NPD at Ipsos. Now, Frank, I do have a question. I have always said Ipsos. I think it’s pronounced “Ipso”, is that correct?


If you are French.




For everybody else, “Ipsos” works fine.


Perfect. So Ipsos is a global market research and consulting firm headquartered in Paris. Founded in 1975, Ipsos is publicly traded and ranked in the world’s largest market research agencies, actually number three, I believe, with offices in 88 countries and employing over 16,000 people. Prior to joining Ipsos, Frank has held senior leadership roles at Nielsen, Greenfield Online, TNS and Lightspeed/GMI. Frank, thanks so much for joining me on the Happy Market Research podcast.


Ah, thank you very much.


You’re speaking at this year’s NEXT event on how to integrate voice in your total customer experience. My second episode on this particular podcast was centric to voice. I am so excited to see a conference, the first conference that I know of in our space, that is really kind of centralizing the communication around this particular medium. How did you first come to realize that voice was important?


If I go back maybe 8 or 10 years ago, when I started seeing people use voice to text, it dawned on me. As soon as people started using features like that, there’s this bound to be an application and research, because they’re just showing preference for a way to communicate. And we have to accommodate those preferences in the way that we capture data. But I guess the real big thing was certainly when Siri was introduced by Apple –I think it was 2011. That really seemed to show the promise of what you can do with voice communication. And it showed that with work eventually that could become a major component of how we collect research data.


Yes, for sure. You know, it’s interesting 2011. I think it’s September 2016 that Alexa launched. I think I have the year right. Does that sound right to you?


That sounds right. Yes.


So the big head start that Apple had inside of this space, and yet they certainly have taken a back seat from a growth perspective with Google Home now being the fastest growing voice-based platform. And obviously, Alexa. I think Alexa is still dominant.


Yes, well, again, a lot of people are using the Apple, the Siri on their other devices. So it’s not just the voice assistant devices that we should think about because research could be captured probably more commonly through the voice assistants on the phone.


Right. Yes, yes, for sure. I was talking to my kids. I have a bunch of kids, some older ones and some younger ones. So the younger ones are utilizing, and this is all just organic, Alexa right now to play hide and seek. It’s just really funny. Oh, Alexa, stop! Sorry. She’s now going to be called “A”. No, no, stop!

So the other side of it is, my older kids were talking about how it would be really cool if they could get an audio feed that highlighted stuff coming out of their Instagram account that they could then somehow magically connect right to their phones, to get in the visual pieces there as well. It is almost like a morning notification or update or what have you. So it’s funny how you think of them and “hide and go seek”, which is very much a tangible, visual-based game can be and has been created in a voice-only context. The other piece that I think is interesting… You mentioned Siri in 2011. I got my daughter an iPhone. She was 10 years old at the time, which I know it’s early, but don’t judge, and we’re driving, and I asked her: “Let’s talk about your best friends.” That sort of dad conversation, right? She said it was Hannah, another girl named Emma and Siri. She is actually pretty clever, my daughter. At least, I’d like to think so. But anyways, she actually told me in the car that, she was all in, that Siri was a real human being and just happens to be constantly available to answer questions that she has. And of the top people in her life, one was an AI.


Wow, that’s an interesting story. It is true, there’s people that are interacting with these devices constantly, many times during the day and they’ve customized the voice to their particular accent, to their gender, to their favorite sports player, whatever might be.




And it becomes a pretty indispensable component of their day-to-day life.


Yeah, for sure. And I think that as we pull back the user experience, and I’m not asserting your age on the podcast, but I didn’t grow up with social media by any stretch of the imagination. Libraries were the source of information for me, not the Internet growing up, and so my context is completely different from my kids. My teens, who have grown up with access to social media during pretty much their whole awake life. And then now my youngest ones, my two and three year old, who are interacting yet a whole different way with technology and access to entertainment and information. As leaders, we need to make sure that we have, that at the industry level we create the humility to understand that here’s a massive evolution that’s happening in these other two generations that we just don’t exactly connect with.


Yes, that concept came out a while back about the digital natives and so forth and much what we call the youngest generation because they are even more native. But it does also suggest that across the other generations, there are some that maybe are much less amenable to these technologies. And so it means that when we create research services, we have to really cater to quite a range of capabilities in the technology space.

In many cases we have to use multiple modes of data collection, which each have their challenges, and they may be well suited for a certain generation and not so well suited for another generation. So you have to have a secondary methodology that you have to try, and make the two methodologies fit together somehow in order to capture the representative data.


It’s kind of like an analogy: being we went from paper to Internet, this whole transition in market research. Obviously, there’s still some surveys that are paper-based but predominantly Internet is taking that space over. And then, later on, in 2006, the release of the iPhone smartphone. So now you’ve seen, from 2006 to 2010 again a migration. I think the majority of surveys now, over 50%, are taken on smart devices. Are you seeing voice is part of that narrative?


I could talk about several different waves of innovation, and how researchers have followed the adoption, which was curved from… I worked in postal. To go back to those era. I worked in postal and CATI and face-to-face. And so most of those have transitioned to the online world, and then with mobile coming in, moving from computers to other devices, tablets for a while as well. But they seemed to be phasing out, but now the majority of collection is now on mobile, indeed. And within these devices, such as with mobile, with new technologies being available to move to voice, this will be an equal challenge and an equal opportunity, shall we say, to capture data in a new way. And like the other challenges, we made some big mistakes. When we first moved to the online, everybody’s heard the story about you just took CATI surveys and telephone surveys, and adapted them to the online, but didn’t really make use of the unique characteristics available online to the extent we could have. And then the same story was really true, it took a very long time to rethink how our surveys could best fit in the mobile environment. And it’s really just getting to that point now. And I’d like to think that at some point we learned from all these lessons and say:  “All right, well, you know, there’s going to be a sizable portion of data collection in the future, which will be voice-based. And let’s plan for that. Let’s figure out how to leverage that new methodology to its fullest from the very beginning and not kind of drag our feet”, and so forth.

I am definitely seeing enough of these transitions to see that this is going to be a big one. And it’s going to be an important one that’s going to provide a lot of great opportunities. And ultimately, there are going to be companies that do a good job of figuring out how to work in that new space and how to leverage the new capabilities of voice for data collection and there will be others that don’t, and suffer as a result of it.


Yeah, gosh, your point about mobile is really interesting to me. I hadn’t actually considered that, but, 2006, it is about 13 years later, and what you just said is actually very accurate. Actually, I don’t even know if we, as an industry, have completely adopted it. We just fielded a survey that was not mobile compatible, hilariously enough, we helped to fulfill sample in a survey I should say, and we saw a massive dropout rate. We wondered: “Why in hell is the incidence so bad?” It turns out we checked it on our smartphones, and sure enough, we couldn’t take the survey on a smartphone.


There’s still probably about 25% of surveys that can’t be taken on a mobile device. Something like that. It’s still a problem, you know?


Yeah, only if you want representation.


Now, the problem is that the people that are only doing it on a computer are not necessarily representative because there are whole groups of people that almost never worked on a computer.


Ha! That is exactly right. Such a great point. You know, that’s another interesting point there too, and I know we’re a little bit off topic, but is Rogier Verhulst, Head of Insights for LinkedIn, told me the day of the email solicitations is dead, which he’s making more of a point, not like a state of truth. His point is that there’s entire working parts of the organization that literally don’t check e-mail anymore right. They are using Skype, or sorry, well they are using Skype, but they are also using Slack and other platforms.


Yes, Slack.


Exactly. Right. Exactly. So if you’re exclusively soliciting feedback through email, which worked great prior, then you may be missing access to a subset of the population.


Yeah, well, on that point, we see ourselves soliciting responses from respondents through at least a handful of different ways. This in-stream, which is sort of in social media or in-Linked stream there, where you are recruiting people in real time to a survey.  In reality, in augmented reality. And there’s quite a bit of this being done there in social media, in voice, as well as the typical email and other communications methods.


So what is the biggest challenge for market researchers to get consumer opinions in a voice context?


Well, I would say that we can’t underplay the technology challenge of the AI component. What we really need to have is a conversation with respondents and conversation between a computer and a person. And to do that, the computer has to understand what the person is saying and use that understanding to then ask further probing questions. It’s one thing to just say: “Okay, can we take these basic closed-end questions, and get a computer voice assistant to understand them?” And I’d say: “Yes, We are pretty well there on that part.” But that is back to the point earlier. That’s just taking mistakes of, using what worked online and trying it on the next mode when there’s an opportunity to do better than that, and that’s through really moving to a more conversational mode in the voice context. And to do that, you really have to be able to understand what people are saying. And I think there’s this great progress being made in this area, that you can train a database at a category level, you can take social media data, for example, and use that to train a research database, which is much smaller, not adequate for training a database on the terminology used and so forth. But we’re not really there yet for large scale conversations, shall we say. But that’s where the ultimate goal is to me. As we progress with AI and with the natural language processing, we’ll get better and better over time at being able to make sense of what people are telling us. And based on that, asking the right questions to follow up on that.


So in 2023 it is projected that about $80 billion dollar will be done on voice, which is obviously a material channel, for whether it’s Walmart or Amazon or whatever, Google, etc. And that’s around the corner, so to speak, when you’re 48 like I am, it feels like it is pretty close. Why do you think voice isn’t a bigger deal in context of insights right now?


Well, it is getting a lot of voice, a lot of mentions at the moment. It’s got even more buzz. And, like I say, augmented reality that I just mentioned earlier is probably an even bigger industry than voice is today. And yet it’s not that important in a research perspective. Part of it is that I think research companies are pretty good at adopting, and using new technologies, but we’re not that great at developing them. So we need other industries that come up with these new technologies to come around and come into our industry, and help us figure out how to best deploy those technologies. The big industries for voice are things like automotive, or security, or financial, or retail. They are the early movers. I think research is a bit smaller, and the technology companies will be focusing on research as an opportunity in the near future. But the challenge is here. Technology is a bit more too complex for most research technology companies to try and tackle and optimize for research purposes.


To your point, do you see that as more of a partnership opportunity for a company like Ipsos? Or is it a hybrid where you’re going to be developing your own suite of solutions?


Well, the industry tools are going to be pretty good. I don’t know that many research companies are going to be able to tackle, to partner with the likes of Apple and Google and such realistically. But they’re making good tools available to be used. We have already been doing stuff within Amazon and so forth where you could get a skill picked up by those applications so that you can get surveys in there. And we’re going to have at the NEXT conference some good demonstrations. Myself and my co-presenters will be showing several examples of real projects that we’ve run this way. I think there are tools that we can leverage. I think that they will be building blocks that people will build upon because again, we need wide availability of these tools within whether it’s Siri, or the Google voice Assistant, and so forth. We need these things technology to be deployed out there and then we just have to make use of it.


Yeah, right. The analogy being for me, if you look at whether it’s Facebook Ads that have A/B testing baked in or Google Analytics, obviously, which is to show you a nice point of view of what’s happening in your website or apps, one of the things that market research is doing a good job of right now at the brand level is tethering that data to state consumer opinion data, and then creating more context for the business insight. It sounds like what you’re saying is voice is going to follow a similar suit where you’ve got these building blocks, whether it’s AWS or what have you, that will empower the bulk of the platforms, and then specialty pieces whether it’s an injection of the lady whose name starts with “A”, who I don’t want to be in on the interview, she could follow up with the: “What do you think about that last purchase?” That sort of structure.


Yes. I would say that we don’t need to create everything from scratch. We just need these tools. Everybody has a mobile phone. Many people have voice assistants. We just need to leverage the technology that already exists within them. And then, we can layer on other technology. In the case of the voice assistant, you’re not getting actual voice, you are getting the transcription of the voice, but I think Video Analytics is on the rise. And I think that’s going to be a big type of voice data capture that will be very important to the industry. Because I do think that the sentiment, emotion and voice tonality and all those different things that you can layer on there can further help you understand your research participants. So I think that down the road, I could see us doing segmentations based on voice, based on what we teased out of a voice in terms of the personality types, and they would be equally as valid as an answer in a questionnaire.


I think you actually already answered this in a number of different ways, but I’m going to really try to kind of hone in on this practical point. What is one practical take-away that participants can get out of your talk at NEXT?


I’m sort of a field work expert, if you will, I have been doing it for a long time. And so what I tend to look at is… And I know that researchers are going to ask a lot of questions about representivity, and how you blend sample, how you adjust the data for different collection modes and those types of things. So I am going to try and at least get some initial answers to some of those researcher-type questions in addition to the use cases that we hope to have several, like I said, good use cases that will illustrate the best ways to leverage functionality.

One thing I didn’t really explain is that like in a diary situation, there is a lot of repetitiveness in the data collection. You don’t want to be asked the same question again, and again, and again. If you have a series of five questions, you need to answer in a diary several times a day every time you do something like the laundry or something, using the AI techniques you can just give an answer for all four questions at once. And the voice assistant will listen to it, think about it, and recognize that you have answered all the questions and just say: “Thank you”. So there are some practical uses like that, that do make the research experience better for the participants, and that’s really what we’re focusing on the near term.


Yeah, I think so. I’m really excited about the point you just made, which is we care about the respondent experience. Research is right now being done at a scale that is unprecedented. Even if you look back 3 years ago, it’s crazy, but even if you look back 10 years ago, or 20, it is mind boggling that we’re doing as many surveys as we are right now. Everybody is doing surveys, it feels like. I got my tires changed the other day. And guess what? The local tire shop, which is not like a national chain, sent me a follow-up NPS survey, which was hilarious. So they are really starting to care. Seeing market research as an extension of the brand is important. And so we need to be better stewards of the respondent. I go back to the eHarmony example. I don’t know if you are familiar with that product.


Yes, not directly but yes.


Right. Prior to when I got married I did use them, by the way, but it was.. it was arduous because it was right when they were relatively new, and it was about a 400 question survey. It was really, really tough, like a multi-day thing. And they were very early in the space. They were able to reduce it to a subset of open ends with just a few closed-end questions and then, from the open-ends did populate the 400 variables that they needed in order to optimize matches. So the interesting part there for me is that they did this in a text-based environment, early leaders in sentiment and such but now you think about a 30-minute survey, and it is really hard to do that. I would argue, nobody could actually do that, maybe 0.25%. But you could actually have a conversation and have that be meaningful and then ultimately populate the data set in a way that can be analyzed for researchers.


Yeah, that’s very much my point of view. I think about qualitative and quant. They are coming together, and I think it’s a good thing because quant people are good at listening. And quant people are really experts at asking questions. But they tend to ask questions with a set of closed-end responses. And what we really need is good questions with open-ended responses where people can say whatever they want, and people that say some things that if we find particularly interesting, maybe we go back and do a return to sample through a voice survey or some other method. It might make it tougher for Data Analytics because it’s less structure data but ultimately, again, it’s getting more towards a conversation, person to computer, where each conversation eventually becomes unique. That is driven by the participant, not by the researcher. That is when we really know we have come full circle from the original days of face-to-face and CATI where it was a conversation as well. But then we put a lot of technology in between and methodologies that, in some cases, made it easier and more efficient for us, but didn’t necessarily preserve the deep insights that we were trying to get. So ultimately, if we can find a way to do things fast and at scale with deep insights, then we have really succeeded. And I think voice is going to be one of those ways that is going to help us get there.


Oh my God. Yeah, for sure. I’ve been seeing this is a growing trend. I was just at IIeX in Austin. There’s a lot of qualitative technology companies that have been entering our world over the last couple of years and really, if you think about it, a survey is just a surrogate for a conversation, it just enables a conversation at scale. And the reason that we have closed-ended questions is because I’m lazy to analyze the data… Through AI what we’re able to do in the sense of analysis, etc., natural language processing, is really, for the first time, be able to have that conversation at scale so that we can get qualitative and quantitative or qualitative insights at quantitative numbers.


Yes. And so what you can do is when you ask that question, you can get a set of responses, and then you can do probing, you can teach the AI to know what to ask.

If they say this, then ask that. It gets smarter and smarter when you train the database at asking what is the next question to ask to get deep behind the meaning of what they have in mind when they answer a question a certain way. And that’s just very hard to do in a quant survey today with the typical survey technology.


So looking forward, not too far, but relatively near term, what do you think is next for voice powered surveys?


Well, let’s see. The real challenge is, of course, that surveys are too long. You really can’t do a 20-minute voice survey. It’s just not going to work. It really has to be more like 3 or 5 minute survey because people just don’t want to talk to the voice assistants in these long dialogues. So I do believe that there is more and more opportunities to break up those long surveys, and to look at them, like you were sort of suggesting, into pieces and ask different people different parts of the questions, infer some of the questions, and so forth. We have to be much more creative in how we approach surveys, asking people questions. Voice could be a very good component of that. But it has to be changed down to, using them for specific situations where you’re hoping to get their deep thinking involved, and then open it in that kind of environment to answer a tough question, not just something that is a simple tick box.  I think that there will be a place found for voice surveys very soon that’ll be just one of the tools in the box to capture research insights. And then over time, I think that we will just find more uses. But I think the initial uses will be returned to sample, like I mentioned, or diaries that every repetitive nature or very short surveys, which could go out for a very large portion of the population. And from that, you call out a smaller group that you want to do in more depth and maybe move them back to an online survey. So I think there’s going to be a lot more switch mode-type things and broken-up surveys into pieces, and a much wider range of interviewing techniques. Back to your IIeX example, certainly the tools around big qual have been coming out at a rapid pace over the last few years. There’s a lot of great stuff there to do more and more survey interviewing. And I think that we will be leveraging them in a voice environment fairly soon.


My guest today has been Frank Kelly, Head of Innovation and NPD at Ipsos. Frank, thanks so much for joining me on the Happy Market Research podcast.


Hey, thanks very much.


All of you, for more information on the Insights Association’s NEXT conference, again that’s June 13th and 14th of this year. Please visit That is  I hope to see there. Have a great rest of your day!

NEXT 2019 Podcast Series

NEXT 2019 Pre-Conference Series – Ellen Kolstø – IBM Q

The 2019 NEXT pre-conference series is giving listeners an inside look into companies such as IBM, Voice Metrics, Ipsos, and Pulse Labs.. Join insight leaders on June 13 – 14 in Chicago for NEXT, where you can discover how technology and innovation are changing the market research industry. In this episode, Jamin Brazil interviews Ellen Kolstø, Design Principal at IBM Q.

Find Ellen Online:




Hi, I’m Jamie Brazil, and you’re listening to the Happy Market Research podcast. My guest today is Ellen Kolstø, Design Principle at IBM Q. International Business Machines Corporation, or IBM, is an American multinational information technology company that is headquartered in New York, with operations in over 170 countries. In 2016, IBM launched the IBM Q Experience, which is an online platform that gives the general public access to a set of IBM’s cloud based quantum computing. Ellen has hosted lectures at the University of Texas on design for artificial intelligence and has served in senior roles on both the agency and services side for companies including JWT, Young & Rubicam, Leo Burnett and BrainJuicer. Ellen, thanks for being on the Happy Market Research podcast today!


Happy to be here! Thank you.


Tell me a little bit about your background. This is kind of helpful for us because it level sets, and gives us a little bit of context of who you are.


Yeah, always a great question. So I started life in the agency environment as a strategic planner and so it came up through that world of account planning. I’d like to say it came over from the Mayflower, sometime in the 80s, from the British, and I grew up in that culture where it was very much about understanding customers and working with them and doing the research yourself so that you could translate that into creative strategy for communications. So I started in that world, and did that for quite a while. Then I felt that over time, the balance of the amount of research that was getting conducted shifted over to clients themselves, and they were taking on more of that in their own realms, and agencies were doing a little bit less of that. And so I found it very attractive to move into the realm of market research, where I could spend all my time conducting research, which is my favorite thing. And that is when I moved into that world and into BrainJuicer, now known as System1. I liked that environment as well because we did a lot of really innovative types of research using technology, so it combined these two worlds that I’ve been playing in, especially most recently. We did a lot of online ethnography and also online community. So you had a lot of tools to use and have consumers come with you for weeks and months in some cases as they work through different experiences with you, so that you could maximize products. And it was really fun, whether it was a long-term engagement, working with them on their relationship to cookies and unboxing experiences or how they selected their phone service and all that kind of fun that went along with that. So I did that for a few years, and then, I had this interesting opportunity where someone said: “Hey, IBM is looking for people with deeper search experience in what we call ‘user research and technology’”. Looking for that for Watson, specifically in the realm of AI, specifically they built up that team because Watson was new three years ago, it was just getting started, especially with the design team, and that is the group that creates the user interface and all of the tooling that our customers use to create AI themselves. I decided to go talk to them, and it was a really great experience. And I ended up there in a completely different realm: total technology, business to business, enterprise environment, but in a completely new and exciting space. And I was very energized by that. And that is how I ended up making my way to IBM through some of the other areas.


Where did you grow up as a kid?


I grew up in Houston, Texas, of all places. So I had actually spent my career moving around and worked in San Francisco and Chicago and Boston, and all these other places. Then I decided to come back to Texas and work in Austin at an agency, and came back to my roots here. And I really love Texas because it’s an amalgamation of a lot of things in this one giant state. You’ve got big corporations. You’ve got rural areas. You’ve got tech corridors and Austin, agencies in Dallas. So it’s just a lot offered here, but yes, I grew up in Texas and decided to come back to the Wild West, if you will.


So I did some digging and preparation for this episode. In 2015, on LinkedIn, you published a long form blog titled “Customers as Mentors”. And you opened with what is probably one of the best quotes I’ve ever heard, and I’ve never heard before, which is pretty unusual. And that was: “The purpose of business is to create a customer who creates customers.” And I thought:  “That is exactly right!” So I know you recently spoke in Austin at IIeX, and then you’re going to be speaking at the NEXT conference coming up in Chicago on June 12th and 13th. What are some of your favorite examples of how AI is helping us better create customer advocates?


Well, that’s an interesting question, and part of my point in that blog was that it’s really great when companies or good companies start to look at their own customers as potential mentors for new customers as in you’ve got all these customers you have a relationship with who’ve been through the journey of adopting your product, especially in categories where those products where there can be a lot of work to adopting them —and technology being a space very much like that. So if you pair them up with brand new customers, and get them started together, and wouldn’t that be a great thing to do? And I think some companies have looked into that, but I think it’s still right for growth. So it’s interesting that when you bring AI into that because AI obviously as a machine has a different perspective. It’s a human-generated perspective because we make these machines right now. But the rule that I think AI can play in that it’s almost becoming that mentor itself because you’re seeing that in a lot of the spaces where AI comes in the chat bot space for the conversational system space where let’s say, it’s midnight, and for whatever reason, you decided to download that new piece of software, and you’re not sure how to do it. And you need help. That’s the time when you may turn to a machine, and AI can help you get through that process, go through that journey of downloading that software correctly. So it ends up creating machine mentors where what I was talking about were human mentors. But you end up having these machine mentors, and they can be as useful and helpful because they’re available 24/7. They ideally, if it’s done well, know the questions you’re going to ask. That doesn’t always happen right now, but it is the vision. The vision is to be able to get the help when you need it, how you need it.


I know you’re going to be a little bit biased here, but who do you see in the space leveraging AI for driving customer experience particularly well?


Well, that’s a great question. I am biased, and it’s some of the folks that we’ve worked with, I will say —I was using that example of downloading software, I would say Autodesk, which is the company that makes AutoCAD and all of that software that helps architects and a lot of people that are doing a lot of rendering. They have a very advanced system that allows you to do a lot of things and get a lot of answers directly through that system. And they have worked long and hard to get a system that’s very thoughtful, that’s very focused on the key questions that customers need and is able to really help them. Now, it’s a different focus in market research. In many cases, we are not always looking at AI right now as being a direct interface to us. It’s more than, it’s a tool to help us in active analytics or insights in your engines to understand a lot of large scale data if you’re a market researcher. At this point, we are not using bots to field for us. Ha! Maybe somebody is. Maybe somebody is trying, but I think we still want to be the one asking the questions. Obviously, you could argue that surveys are an automated form of that, but it’s a different type of research data collection. But at this point, I think AI is in the realm of being a tool in market research, and I would say that it is definitely the best place for it to be right now.


I have spent about maybe a third of my career doing qual and the balance quant. Research is really just a conversation at scale. You don’t need to do research when we only have one customer because you’re talking to that customer, hopefully. But as soon as you are IBM, then we have got a lot of customers, and we can’t actually understand the customer sentiment or put the customer in the center of the conversation unless we actually conduct research and facilitate that conversation. What’s interesting about AI to me is, and you probably saw this at IIeX, that there’s a lot more companies that are entering into market research that are leveraging AI for qual, which is allowing bigger base sizes to be done and historically possible. And when you think about my career, this is way in the 90s, late 90s, mid 90s, we would do things like collages. You have probably these kinds of projects.




And then, we would basically try to put together the respondent collages in a master collage, which is really funny if you knew my art. I never got a repeat customer on that one. I don’t think I delighted customers there. My point is that we were able to actually conduct these kinds of exercises, and then the machine put them together. The AI put them together in a way that is actually meaningful and connects to the audience. Are you seeing that sort of application in market research looking forward? Is that one of the growth areas?


Well, it is. It’s funny that the presentation I made at IIeX was actually around caution with AI.


Oh, interesting!


Understanding where the models are at this stage of the game is not to say that, as I said, you can’t use them or have them be a part of products and services; that can be very helpful. But I’ve spent the last three years watching our customers build AI in their own systems, and seeing the tremendous amount of work it takes to build a really solid, stable model that is reliable, that is as balanced as possible. I mean, bias is what it is, so it’s going to exist but you can get as close as you can. It’s a tremendous amount of effort and work. It’s not something you stand up quickly. It also requires, in some cases, hundreds of thousands to millions of data points for something to be really reliable. Think about if you start as a child and you don’t really know the difference between a cupcake and your dog. You’re not really familiar as a little kid but you start to see that thing over and over and over and over, all these elements, and that’s how you learn. AI is the same way. So you can’t expect after, in some cases, five times an image comes up that AI can correctly identify every time that it’s a Porsche. There are so many elements to a Porsche to get it right, from the shape of it to the texture to the colors to the different elements that are on the vehicle to the logo. It’s got to pick apart all those things, put it back together and identify that as a Porsche. And that’s kind of the value or the promise of neural networks, right? But it takes a lot of work for a model to get that right. And so I was illuminating at that conference, under the hood, how the sausage is made, which is what I will be doing partly at NEXT too just to arm market researchers with an understanding that I think the smart move right now is to use AI but use it with caution, and double check what you’re getting! Don’t expect that it’s a black box that magically spits out the right answer, or that its first passive data is going to be better than what you could do. It may not be, and it takes a while for it to learn from other people, to run enough times, to get things right. And we are at the point where you just have to make sure that your own human intelligence is a part of the mix. It’s not magic. It is very much augmented intelligence, which is what we like to say at IBM. It’s going to add to what you’re doing, but it’s not at this stage going to replace you or what you are able to do.


Yes, I just had a conversation yesterday with Aggie Kush, the Head of Insights he had a lot of titles, he was the Head of Insights for BSkyB. He finished his PhD talking about machine learning. One of the things that he identified going through his thesis, and I think was actually core to it, is that AI in and of itself can reinforce biases that we have, maybe even a gender bias, because it’s recognizing these patterns and then basically playing of the pattern recognition so gone unfettered, it actually could not have the outcome, whether it is social or otherwise, that we might want, meaning that we really got to pay attention to the models and the actual implications of the of what the machines are telling us.


Yes, you play in right into an example I gave at that presentation, which was a study that was done in 2015 around Google Search. Google Search is a great example of AI in use and with a large trained model. All of us when we do search or training that model, right? And this isn’t a dig on Google because, in fact, the way this worked out made perfect sense with what you’re saying. But within their search, university looked to see that whenever someone searched on CEO, they focused on this one instance. When you searched on CEO in 2015, 27% of the CEOs globally were female. But yet when you searched on CEO and Google, female CEOs only came up 11% of the time, which would tell you: “Oh, hey, my model is biased”. Now, Google rightfully came back and said: “Hey, this is based on what people are putting out, whether it is ads, whether it is articles, whatever images they are using, that’s where this is pulling from.” And the university came back, I believe it was Washington University, that came back and said: “Well, that may be true, but we also believe that whatever people are clicking on is training your model.” So if only 11% of the time are clicking on female images, then the model things that that’s the amount of time people want to see female CEO images. And it will continue to under-represent. So it’s exactly the point you made. And it is unintentional bias because that’s the other thing I’ve heard a lot of discussion around: this idea that machines will be able to be unbiased because they’re machines, and they will avoid the unconscious bias that humans have. Well, no, actually, humans are part of the training process. And so that unconscious bias was absolutely present in that example. Nobody was consciously, I believe, trying to say: “I’m going to search every time until it changes its model.” No, it just happened to be that that’s the way it went. And now you have got bias in that model. And that is the other reason I say to always double check what kind of models companies are working with because how much work are they doing to troubleshoot these kind of issues? Are they really looking back at their models and saying: “Oh, we know the types of people that are using our software, whatever we are offering that has AI in it, and we’re going to go back and double check and see how that’s augmenting our model.” Because AI models are never done. You don’t create one and walk away. You are constantly working on it and seeing how it changes because it’s a very constantly changing amorphous thing. So that is where I get on my soapbox about. How do you use it? I still believe it has tremendous promise, and it will always have tremendous promise. But you want to make sure and use your own intelligence in all of this as well. And don’t underestimate your own intuition at certain points.


Do you think there’s some overlap? Because we moved away from the institutional tracker. I mean, not like whole sale, but it’s become a smaller and smaller piece of the corporate budget. You know what I’m talking about, right? The quarter million dollar or million dollar…


Okay. Yes, I worked for a lot of them.


Yes. So those are going away, but at the same time, as to what you’re talking about, I have never heard it cast exactly like that. But these machine learning AI systems are in a lot of ways uncovering the direction of the consumer, which is really one of the big intends of measurement from the trackers. Do you think there is an analogy there?


Potentially? Depending on how people are interacting with AI in the tracker and who is answering the questions, I think there will always be an opportunity to double check what you are getting back as a result of that. Different from a survey, without AI in it, where there is an answer, you click on it and it’s done, AI is always training and because it’s always training, yes, things can change. And so you are just going to want to know how that might change. So, sure, it’s certainly something to keep an eye on for sure.


I think it’s a bad idea now that I hear you answer that question. Okay, so how can modern insight pros use AI?


With caution. Ha! I say that because, again, I believe there’s a lot of value. Like I said, where I get most excited in market research is with Predictive Analytics. I think there’s just a tremendous amount of opportunity. We always struggled with market media modeling. We are always trying to model things to understand what people were going to be doing. And we never had a really great way to at least get an idea of where people were headed. And predictive analytics, especially where AI can aggregate a ton of data, look across many things and start to make connections, will be invaluable. And I think we will get a much more accurate understanding of what could be happening if we were to run certain media mixes, what do we think the outcomes could be. I think that that’s where it’s got a tremendous amount of promise, and I’d be very excited to see how that moves forward.


Yes, I did a fair amount of modeling in my early career. The way that I was taught to do it, which is to say, there’s lots of ways to do it, is you find it, you asked a question in your survey, which is something like “probability of purchasing a TV”, and then that level sets against actual TV purchases over that period of time and so it gives you a baseline. And then you ask another question similar to that but about a new product that your customers are interested in measuring and then, perform a regression. And then all of a sudden you’ve got that or a Van Westendorp or some other kind of methodology that is leverage in order to come up with the predictive… well, Van Westendorp is a little bit different. My broader point is do you see marketing research as a discipline starting to use and leverage AI in order to do these market predictive models versus the traditional, old school stats point of view?


I would say it is probably being more valuable in that space, for sure. We worked on so many regression models, and I still couldn’t tell you if I really knew if any of that was going to play out. It was hard. There is a famous quote… Oh, gosh, I’m not going to get this right. Something about “I know half my advertising works. I just don’t know which half.”


And then we categorize that half we don’t know under branding.


Right! Exactly! And it’s never going to be a completely exact science. I think predicting behaviors is very hard. But statistically, it still was not quite enough of an indicator of what was really happening out there. AI has the ability to look at a lot of things and because it can also look at unstructured data, you have this unique opportunity where it could look across more than just the statistics. Now, it can look across conversations and different things that can be fed into the whole pie and tried to get a better understanding of what could potentially happen. That’s where AI’s promise has always been and that it has now so much more data to draw from to try and find these answers to very complicated questions.


AI is part of the tool kit, right? And let’s say that you’re entering into the insights role inside of an organization, marketing research or some other some other way. Well, actually, let’s focus on market research, what skills do you think the person should be cultivating in order to successfully drive inside of the firm, basically informing the executive level business decisions?


Yes. There’s a lot of different things. So the first one that came to mind because it is the one that I constantly run up against is flexibility. You have to be willing to roll with what comes along, not only with all of the changing technology and the different things that come up, but it can be very difficult to leave sometimes your opinions at the door and say: “OK, well, let me look at this a little bit differently”. Insights? When you get to the executive level too, they need to be pretty battle tested, right? You want to make sure that you feel pretty good about them, which means you have to at some point vet them in various different ways to know that you have something collectively that you feel is going to stand the test of time, especially the enterprise, where big, big, big decisions get made, right? And so you have to be flexible, the tools you use in the kind of data you’re looking at. You have to be willing to look across a whole bunch of different types of data, trying different methods. I don’t think you can do “plug and play” anymore. I mean, I think I i’s back to your point about all of those longitudinal studies, and all these tracking studies of “there was one way to do it”. You did that every time and you reported that number at the end of the year. And now, there’s so much innovation and change. I think staying on top of it is challenging. But I think also being willing to be flexible and reinvent at various times is going to be a really important skill set.

I am also going to go back to, and this feeds into flexibility a bit, creativity, which is also super important. And it’s a funny thing because I think what really helps that is to be able to draw from things that aren’t all related to what you’re doing or even in some cases, your domain, right? It’s looking out what completely different companies or different competitors are doing or even people completely outside of the industry that you are in, and trying to see how you can maybe utilize some of those elements in what you are doing to try and come up with new ways to think about things. Every industry is getting so incredibly competitive, certainly saturated with a lot of known insights. Getting something new and different is just requiring a whole other level of flexibility and creativity and inventiveness that you are just constantly having to hone, and it’s not easy to do because you’ll get in myopic into your workflow and then go: “When was the last time I even read anything on a new technique in this area?”, but it’s something to keep in mind.


This is such an interesting point to me. When I started my career, it used be the case that it was adequate to conduct a consumer survey and then analyze, PowerPoint, and then story-tell, right? But it was all in the context of that study. Now it feels like that’s wildly inadequate, right? You need to really hone in on providing the context, market, business, social, whatever, of that particular insight because the context informs so much of the implication of the data. And so one of the things that I’m seeing more and more in research reports is that maybe 25% is spent on both the setup, the context, and the implications at the business level. So it’s almost like we’re moving a little bit broader, and then also going deeper with the insights.


Wow! It’s so funny that you mentioned that because context is a big, big thing with me. I completely agree. It is telling stories, and it’s telling stories with the details where you can really start to see what’s happening. And I think in on the side of technology, especially with usability, there has been a tendency towards scores and just very almost quant-like representation of the learning. And I have pushed to put a lot more context even around that kind of thing. Just because somebody is navigating through a website does not mean there is not a lot of interesting things, especially if you are sitting there watching them, that can tell you about their thought process or why on that day, they ended up in certain parts of the experience. And that is where it gets interesting. It’s also true that your insights are better remembered with context. Without context, they are “somebody wants that”. But when you can go back and replay a story to somebody else about the context of why they want it, it gets institutionalized, it gets internalized, it gets retold and it’s that whole fireside chat kind of phenomenon. I’m a big believer in context. I would almost say that the context is 90% of it. And I completely agree with your point.


What I described is actually incorporating a lot more data really into the narrative that you build out. But the master, storyteller, they’re doing that. But now they’re actually the content on the slides, and the actual story that they tell is re-tellable. So it’s actually a hell of a lot less content that winds up getting displayed, and the story is profoundly simplified to its core essence. So it’s really interesting; it’s a much harder job today than it was before, I think. It is one of the reasons we have to leverage any tools that we can in order to help us.


Yes, and that’s where again unstructured data comes in, right? It is all of that kind of conversation. It’s interesting how AI will be able to help us with that. I think insights engines will get a lot better, and they will start to be able to serve up that context in ways that we can’t possibly get through all that data, and that will be super exciting when that happens, and that all of that context is that we want to hang on to.


Yes, insights and context, that would be interesting business to start, I think.


No kidding! That would be great, right?


I think I am doing about 1,000 interviews, and I’ve said the story before on the show. So I apologize to the listeners about the redundancy, but it’s rare. It’s worth mentioning. I did a quant study, relatively short, and then at the end, I asked: “Please do me a favor, and take a 15-second video or some period of time video of your environment”. And one lady, I’ll never forget it, took a video of a number of kids where they were running around like chickens with their heads cut off, as my mother would say. And I was thinking to myself: “All of a sudden it drew everything into question about the insights that she was providing in that survey for me.” You know what I mean? It feels like… totally. That was really important, the context of her providing that insight, which in that case was potentially moving a multimillion dollar ad buy. So it seems like maybe they’d want to know that? I don’t know. Anyway.


Absolutely! I did mobile ethnography, like I mentioned in BrainJuicer, where we had customers videoing various things, unboxing experiences, as I mentioned, in all sorts of things. And you saw the context there of their world. Right? There was one really funny when I was doing on a cookie that was being introduced. The husband was more excited about the cookie than the wife. And the wife was the one in the study, and he kept creeping into the video and taking it, and she eventually had to hide the box from him. But it was an interesting dynamic that you want to say. And the cookie was targeted to women, as they can be because it had a certain dietary benefit. But it was like “Who cares? See, this guy loves it.” So, yes, there’s so many stories that could be told by being in that environment, obviously the power of ethnography and the power of storytelling.


Yeah, which links to where you started, that is, the power of AI because it’s so hard to do that at scale.


Yes. It is hard to do that, yes. Yeah, it is the promise of it, and it will get there for sure, and it will change everything. I still firmly believe that even as it starts to be able to go through a lot more of that data and comb through it and give insights, I think humans are still going to be very, very much in the mix with it in terms of building off of it. You know how you probably collaborated with another researcher before, and you have kind of rift off each other to come up with the ultimate viewpoint on something or the ultimate insight. I believe that is how the relationship will move forward with AI.


Oh, I completely agree. This whole fear around AI removing jobs in the least, in the next 50 years, maybe 50 years, but not in 20 years, at least not from my vantage point, it’s all about partnership. I liked your augmented intelligence point of view.


Yeah, I agree. I just don’t see that happening.


So on a future look, how are we going to be different as an industry in five years?


Oh man. Well, let me get my AI together, and I will tell you. Ha! Where’s my predictive analytics? I will give you one viewpoint I’ve been thinking a lot about. And this is because I am in technology now and more so, in this space. But I think your UX research and market research are going to morph because I am already seeing in the realm of usability and user experience, all of that research, a lot of researchers in that space saying: “God, we need to understand more about the market. We need to do more up-front qual”. And then, when I was at IIeX, they had several sessions on usability, which was pretty funny, because some of us from the team went to that conference and they said: “Wow, they introduced usability like it was a new technique.”  I think it’s pushing into the realm of market research to say: “Hey, nothing is stopping you from wanting to dig deeper into the online experiences of your customers even though you might be at the brand level, right?” So I think we’re going to see all of this come together as one big realm of customer research, and I think it should because customers will engage with you all over the place. And why wouldn’t you have one researcher, a team of researchers looking across all of it, from the market to the online experiences to everything else in a meaningful way that doesn’t separate out user experience from market research.


We have addressed this next question, but I’m gonna ask it anyway, just to see: If you were going to create a company today servicing the industry, the insights industry specifically, what problem would you address?


Yeah, I like your context one a lot. So this is what I’ve been thinking about for a while, and I don’t know if it’s controversial or not, but it’s this whole idea of “is bias really a bad thing?” The reason I say that is that in research we are constantly saying you can’t be biased, we got to be unbiased, and we all know that’s impossible. You want an unbiased sample, and this and the other will. The panel probably already has biased from a million different angles right that you have drawn from. We know as humans, bias is inherent, certainly there is bias you absolutely want to be careful of, anything that harms anyone. But in some cases, bias is to be learned from. And if it exists, how might we learn from it and gain insights from the bias itself rather than treat it is something we just should either ignore or pretend, we have maximized it out of the equation. So for a business to understand how we can work with bias rather than avoid or against it, I think could be really interesting to figure out. Even with that Google example, there’s more going on there, with how people are clicking on those CEO images. What is it? Is it purely gender bias? Are there other things at play? What can be learned to unpack some of those elements that will help us better understand the role of bias? I would also argue, in some cases, bias is not any different from having a hypothesis. Having a hypothesis means I have a point of view on something without all the data. And I am biased in a certain direction because I think this might be what is going to happen. And then I will go into a study with that hypothesis, and I will obviously look to see that plays out. But we all know you are looking more for that particular other things because that is where your mindset is. It’s not a bad thing. It is something we all do. But how might we think about how to reframe the use of bias in a way that we can learn from it, that we can improve the outcomes and treat it as something that is a part of the mix, not something that we just should avoid.


Yes, it would be fun from a start perspective, it would be really fun, and it’s useful to think about… You are familiar with Myers-Briggs, of course, or whatever personality profile thing?


Yes, yes.


So, like, for Jamin Brazil, what biases do I have in my life that I probably honestly just don’t know about, that are just a function of culture and context?


Absolutely. Yeah.


That would be a really interesting… I don’t know how we would do that, but it that does seem like something AI could address.


That would be a great Myers-Briggs. You are right. Because then that’s something you would know going into any future work. Okay, this is a mindset I’m coming in with, and now what do I do to either to mitigate it or to in some ways celebrate it. Because it’s a funny thing too: I was reading a Harvard Business Review article recently that talked about how employees get reviews, and so many times, reviews are a negative experience because it focuses a lot on your weakness. “You should be doing this.” “You should be doing more of that” instead of “Okay, let’s celebrate what you are good at and find other things for you to do that celebrate this thing that you are good at.” So it’s kind of that same idea. How could you take what might seem like a negative and say: “Well, there may be ways in which this could be extremely helpful with certain studies”, “Having this viewpoint could really make me the best researcher for this type of research” as opposed to “Oh, you are biased in a certain direction, and now you’re not good for certain things.”


Yes, totally. It is such an interesting point of view. I can pick on my grandfather here, my late grandfather so, I will tread lightly. But my point is that he grew up in a World War II generation. And there was just a completely incorrect set of biases that were ingrained there, not in a positive way. I am not saying he was part of some terrible group or anything like that, but it was just different, really different. He didn’t fit into a millennial culture, how is that? And yet, with no malicious intent or anything along those lines, it was just the framework that he understood and agree in incorrectly. So the opportunity for him to get informed on that, to hear: “Hey, these are the things that you know you have inherent biases” because you can often find see them in other people, but they don’t really not be able to see them themselves. And that’s the point. It’s hard to see the blind spots in ourselves. Something like that could be really interesting.




Sorry about this. I totally went away with the conversation.


No, what is interesting about your grandfather, too, is that, who knows? His perspective might be getting smaller and smaller and smaller as millennials grow. So that maybe a perspective that’s also interesting to understand or potentially having a certain study, where there is another angle to things, you know what I mean?


Totally. Out of micro level in and a man macro level, start seeing how that plays out. That’s so interesting. All right, my last question: What is your personal motto?


Ha! I guess the one that comes closest to encapsulating me is: “Always be prepared.”

I learned that a long time ago for my father, who approached everything with a lot of preparation, thoughtfulness. He had he had a plan for everything, and it really served me well of just having some level of preparation is, I think, sometimes 90% of the job, 90% of the battle, whether you’re reading secondary research ahead of a study or you are just getting smart about an industry or you are having a conversation with some stakeholders. Before you get started with something, you have got a good jumping off point that means you are not just going in shooting from the hip in many cases. I’m someone who likes to have a level of preparation. So it’s ironic because in some ways AI is very much about that. Building models is very much about a tremendous amount of preparation going into any kind of work that you are then going to do with it. But yes, that’s my thing. I like to be prepared.


I love that. I got to end on two stories to that point: A good friend of mine, Jennifer Crawford, she took a bet on me when we were at Decipher in the early days. She is the owner of a New York-based research company called Research Solutions. And I remember I co-pitched with her to Meredith about an online diary, something you’re really familiar with, and in that pitch she came in with a folder that was about 0.25-inch thick of preparation. There was a bunch of stuff in it about the meeting. And so we left after 45 minutes. I don’t know if we actually opened it. Maybe we got through two or three pages in the binder, or the folder. And this is the only time I have ever heard a customer say: “I want to thank you so much for being so well prepared for this meeting.” And we won the business. It was a windfall for both of us, the firms. It was spectacular. Anyway, sorry about my reminiscing. But preparation, as it turns out, I think it’s really important. Oh, and the second one I want to mention is Voss Media. Voss Media, which is a big company, is inundated with papers about states of industries, etc. And they actually subscribed to an AI-based system, which does the processing so that they can reduce all this vats of information into a quarry string and pull out the pieces that are relevant and say that they have 99% coverage on their content. So anyway, yeah, I like the preparation point. Thanks so much for sharing that.

My guest today has been Ellen Kolstø. Sorry about that hick-up. Ellen Kolstø, Design Principle at IBM Q. Thank you, Ellen, for joining me on the Happy Market Research podcast today.


Thank you. It was lovely being here.


Everyone else, this is in conjunction with the upcoming NEXT conference. You have a couple weeks still to register. You can find out information online, of course, at as well as Google Next, and it is located in Chicago, on June 12th and 13th, I believe. It is going to be a wonderful event. I hope to see you there as always. I love your screenshots, feedback. Share this. It’s appreciated. Have a great rest of your day.

NEXT 2019 Podcast Series

Ep. 218 – NEXT 2019 Pre-Conference Series – Dylan Zwick – Pulse Labs

The 2019 NEXT pre-conference series is giving listeners an inside look into companies such as IBM, Voice Metrics, Ipsos, and Pulse Labs.. Join insight leaders on June 13 – 14 in Chicago for NEXT, where you can discover how technology and innovation are changing the market research industry. In this episode, Jamin Brazil interviews Dylan Zwick, Chief Product Officer at Pulse Labs.

Find Dylan Online:




Hi, I’m Jamie Brazil, and you’re listening to the Happy Market Research podcast. This is a special episode that’s connected to the Insights Association’s NEXT conference in Chicago, that is, this June 13th and 14th. My guest today is Dylan Zwick. Dylan. I said your last name, right?


That was correct. Yeah, Dylan Zwick. I’m always dead last in the alphabetical order.


That’s funny. I was always first in photos because I am 5’8’’ so we have that objective position. Dylan is the co-founder and Chief Product Officer of Pulse Labs. Pulse Labs is a solution that enables users to launch and gather consumer opinions via voice devices such as Alexa and Google Home. Dylan, thanks for being on the Happy Market Research podcast today!


A pleasure to be here. Thank you so much for having me.


You are speaking at this year’s NEXT event on voice. When did you first realize that voice was important?


So I first realized that voice was going to be big back in 2016 when I bought my first Echo. So I started playing around with Alexa, and realized that what had been the dream of science fiction now for decades, you know, the ability to speak and actually have a conversation with a computer was actually becoming science fact, you know, it was it was becoming reality. And so, I played around with building my own Alexa applications and started exploring the tools that were out there for developers and designers for Alexa application, and for voice applications more generally, and realized that this was going to be a huge space and also that I really wanted to be a part of it. So that’s what got me initially involved.


Yeah, I mean, Alexa in and of itself is really interesting. One of the things that I think is… If you pull back, YouTube right now, I forget what the data is, something like 60% of the Internet is there. It’s a massive amount. And if you look at the bet that Google placed when they did that acquisition, it was, they consolidated the different product lines into a single thing, and then they centralized the KPI to one centralized point of focus, which was the number of daily videos unloaded. And that created so much focus from an R&D perspective that that was all anybody cared about. It wasn’t predicated on revenue or eyeballs or anything like that. That was it. And then subsequently, of course, that was the tail that wagged the dog. Amazon is actually doing the exact same thing with respect to Alexa. I mean, my kids… My 12-year-old can create an Alexa skill. It is crazy easy how they have made the development side of this accessible.


Yeah, that’s been a huge focus for Amazon. And the Alexa team is to open up as many tools for developing essentially applications or, as they call them, skills, on Alexa and then, trying to provide these and encourage as many independent developers to build skills there as they can. So you have a ton of skills, actually, that have just been built by independent developers, and then also a bunch of skills that have been built by brands or professional agencies. And there are even companies out there that are focused exclusively on the building of Alexa skills. And then yeah, you’ve mentioned, you know, they’re also very interested in providing tools that makes this as easy as possible. So you even have a blueprint tool that essentially lets you quickly create a standard but personalized skill without the need to have any programming background at all. And I focused on Alexa and what I just said but Google is also pursuing a similar strategy, and that you can also build applications on Google’s Assistant. They’re called actions, and they’re really trying to build out, and expand, and encourage that ecosystem as well. So all the major voice players and, to be honest, Bixby and Cortana are also very interested in that. So all the major voice players are really trying to provide a platform for as many content creators to participate on as they can.


So the Bixby thing was interesting, right? They launched… I think it was last year. It was Samsung’s voice device. And Cortana. It is interesting to me that Cortana and Siri haven’t had a more dominant role in voice so far, especially considering the head-start that Siri had. Do think that developers are going to need, or should I say, brands are going to need to deploy across all of the major players? Or is it… I may even roll it back a little bit further. I don’t know how old you are, but I was in the Silicon Valley during the whole rise of the dot-com, and there were probably 12 searching engines.


I remember.


…like Infoseek and You had this… exactly! So you had to really pay attention if you wanted to get visibility on the Web in terms of where, what the users… do you think ultimately it is going to be one ring to rule them all?


So right now, it’s certainly a duopoly. So right now, most of the market share there is being taken by Amazon and Google. And so if you are a brand and you are building an application for voice, most of the time you’re going to want to… Brands are interested in actually just building on both of the platforms, and it tends to be pretty easy to port applications built on one from the other. Once you’ve built, for example, a Google action or Alexa skill, translating that over to the other platform —as I said, it’s not trivial, is a whole bunch easier than building a new one from scratch. So because of the market share that both of the major smart speaker players have, most brands when they build a voice application are interested in building on both – and it’s becoming easier to port that so it’s becoming less expensive to do both, kind of at the same time, so most are interested in doing that.

In terms of the other voice assistants, that is, Cortana, Siri and Bixby, they’re all making interesting plays, but they are mostly not competing directly with Amazon and Google on the smart speaker market. So Cortana is actually positioning themselves as much more of an enterprise voice offering. So the idea would be that Cortana would be kind of your voice assistant in the office and sort of the business aspect of voices system. And then, Bixby has an offering that is very tied to your phone, to your Samsung products, though, so it’s really tied to what people are doing on their smartphones. But yeah, I would say that we will see how the future shakes out in terms of who is going to be dominant. I don’t think it will be one single player, but I also don’t think it will be five or six.


That sounds like your framework is really centered around use cases and the context of the interaction. So I have my Samsung TV, of course, and similarly, I have got my Alexa sitting there, but it’s actually funny. So I got Bixby on my Samsung TV that I set up. But I still use Alexa on my Samsung TV. From an interaction perspective, it is kind of funny.

(Oh, sorry, Alexa, stop!)


Ha! I’ve had that happen many times. So yeah, that’s really what you’re getting at is the fundamental goal. The reason that these major huge tech companies are so interested and invested in those platforms is not because they really want to dominate the speaker module. But it’s not because that the clock radio market was so important to them that they’re just going to go in there and crush that. What they really want to do is they view voice assistants as being the operating system of the Internet of Things. So you’re not just going to be talking to your smart speaker, or even just to your phone, but also to your car, to your television, as you brought up, to your refrigerator… I mean, you’re going to be talking to all of these different electronic appliances, and it’s going to be ubiquitous and the primary means, or at least one of the primary means of interaction, will be via voice. So that is essentially the big dream there. Whether it’s going to be something that’s dominated by one particular company, or whether there’s going to be maybe just some underlying framework that isn’t owned by anybody but that everybody kind of builds on, and that maybe you’ll be able to access Alexa or Google Assistant or whatever voice assistant you want, from any of these touch points, will be interesting to watch as it develops.


I mean, what I’m finding so fascinating is the way that we interact with voice. Alexa, for example, has skills, and I forgot what you said, Google Home’s reference was.




Actions, right. So it’s a very much of a human interaction, that is, it is that part of the UX experience. So I could see a scenario where you could successfully address Cortana, and to your point, in a business context, and then, similarly, I could use Bixby for maybe my refrigerator or my appliances or what have you. And then maybe at a personal level, I just want to go ahead and interact with the lady whose name I won’t say. That is really interesting.

And I love how you started out talking about the science fact because being a geek and Star Wars nerd, and Star Trek fan, the way that both of those environments projected the future, it turns out that Star Trek was right with this voice AI always being part of you.


Yeah, and sort of as a side note there, Jeff Bezos is famously a Trekkie, so Jeff Bezos is actually famously a big Star Trek fan. And the Star Trek or Enterprise computer was actually part of the inspiration for Alexa. So it may not be entirely coincidental that they seem to be similar. From what I understand, the Star Trek computer is actually part of the inspiration behind Alexa.


Oh, that’s so interesting! All right. So you’ve worked with several firms, many firms, including today’s top firms, on voice application. What has been the most exciting aspect of that? And then also, what do you see as one of the larger challenges in this early stage?


Yeah, absolutely. So the most exciting thing about it is that voice has the potential to be, essentially, the lowest friction form of interaction between a person and a computer, and also the most essentially natural and intuitive one. Speaking conversation is something that we learn and understand almost innately. There are parts of the human brain that are just specifically wired to communicate this way, and so voice interface is something that, if done right, is going to be the most intuitive and easiest type of interface that anybody is going to be able to use. And you have actually seen that with the… I remember six or seven years ago, a friend of mine’s young child walked up to a television and started touching it, and the television did not respond. And the child thought that the television was broken because he had become so used to touch interfaces. And even before touch, we’re seeing kids that are talking to their smart speakers. And so it’s going to be expected that any sort of technology that you interact with, you’re going to be able to talk with. And if you can’t, it’s going to seem broken. But the big promise there is that, as I said, it’s a super low friction way of interacting with technology, and it also is a form of interaction that can take place when you were otherwise occupied. And so a couple of examples: the big one that’s been so successful right now are things like requesting music to be played, just saying “A word”: “play Despacito”. Or asking for the weather or any of those kind of quick functions that you want to do every day and that can be made really easy and low friction. But what I think you’re going to see is most of the audio consumption is increasing rapidly. People are listening to music and podcasts and radio broadcast or radio shows more and more on digital devices. An so the ability to kind of interact with what you’re listening to via voice is extremely promising because usually when you are listening, you are doing something else. So if you are a marketer, and let’s say that you have an audio advertisement that’s playing on Spotify or podcasts or something like that, the ability to just say —if you want to know more about this, say: “Alexa, tell me more”. And have that instantly send you an email that will tell you more about what was being advertised, and then take you right back to what you were listening to, I think it has enormous potential and power.

Another scenario, another context is driving. So if you are driving, you are in an inherently hands-free situation, your hands should be occupied, so not hands-free because your hands should be occupied. And so it is an inherently audio scenario in which you are able to, for example, order food on your way home for pick up from a drive through via voice in your car, I think has enormous potential to kind of transform a lot of those flows.

What challenges there are today? I would say the biggest challenges are discoverability. It can be hard to really know what is currently there and available, and to remember what skills you want to invoke to do what. So that has been an issue. And then there are other scenarios, other interactions that I think to be the best way to input information, but are not necessarily the best way to get information back. So if I asked for a list of the 10 most popular movies from last year via voice is probably the easiest way to request that information, but then having something come back and say: “The most popular movie was X. The second most popular movie was Y”, etc., might not be the best way to get that information back. So something like a list might make more sense as a visual response. And I think that the combination of voice and audio with visual, so opening voice up as one medium through which you can communicate, is opening up a lot of new possibilities. I think that multimodal is going to be a major part of the voice applications in the ways that we use voice over the next few years.


Yes, for sure. And I think from a researcher’s point of view, thinking about the opportunity for ethnography to be done even though you do not have the video component tethered to it, but always on a feedback option is really, really powerful. If you are thinking about CPG-type or products, whether they are software or service or whatever, that we interact with or real things, then you can always provide feedback as long as you have that particular device handy.


Exactly, and so…


And you could do that, to your point, before while multitasking. So you have the new Alexa Auto that is a really interesting —I think they’re doing a limited release right now. So it’s the in-car version of the Dot, or whatever, Echo. So the closer that you can provide feedback to the actual experience, the better the data, the less the time degradation of the insight.




And right. If you think about being in the shower, and Head & Shoulders wants to do a new product test, and I’ve got my device inside of the bathroom, I can actually provide feedback on that experience while I am in the shower, where before that was just always impossible. You could never garnish that kind of information.


Exactly. So something like the ability to quickly provide, like a net promoter score or rating and then some quick feedback or data about a particular experience can be done very low friction via voice. You could have something like, on your Head & Shoulders bottle or something like that that said: “To provide feedback or rating, just say this particular thing to Alexa and then answer two questions”, which is something that I think people would be much more likely to do than say: “Go on a website and fill that out.” Or if you have the ability to say yes to the bottom of their seat. Say you can use Alexa to provide this feedback, and then we can maybe even send you an email coupon or something like that.

And then you mentioned CPG. Another big, exciting possibility here is that CPG most of the time people purchase is replenishment and reordering. Traditionally, packaging has been mostly geared towards standing out and convincing the consumer to make a particular purchase while that purchase is available on the shelf, and competing with other similar products. However, with this huge shift that we are seeing into purchasing online, I think packaging might even be somewhat rethought as a way of convincing consumers not necessary to make a purchase but to reorder. And the ability to say, for example, let’s say that you’ve got a roll of paper towels, and when you’re done with the paper towel roll on the actual cardboard roll itself, it says: “To reorder this, just say Alexa XYZ”. And it could be just a quick two-turn interaction or something like that, and you would have a replenishment of what you just finished on its way. I think that has enormous potential for tons of consumer packaged goods.


Oh my gosh, totally. I have never heard that example before. Thank you for sharing it. That literally blew my mind. This is going to be the headline quote by the way of the episode because one of my big challenges in moving to a voice consumer journey is that it’s an invisible journey. So the opportunity for a brand like Scott to intercept the point of purchase is quite literally zero. It’s all about my brand affinity, which at the end of the day, paper towels are paper towels for me; maybe not for other people but I don’t particularly care as long as it does what I want it to do. And so, if you can get that brand into the speaker, paper towel ring thing or whatever, now all of a sudden you do have an opportunity to create that connection with the consumer.




And this is what’s interesting: you could actually spawn the transaction because it’s a voice-based trigger


You could just say it right there at the moment where they are thinking: “OK, I need to reorder”. You could just be instantly there, and it’s the simplest transaction it could be. It’s basically just this thing that I have, that I’m out of, I want to re-order replacement of exactly this thing. “To do that, say exactly this”. And it will happen.


I think we should scrap everything we’re working on. And that is the direction…


Ha! This is what I thought at the end of 2016. I thought: “All right. I got to scrap what I’m working on and pursue voice.” Because exactly, it’s things like this that got me and continue to get me super excited about it.


So, for our listeners, is Pulse Labs’ website, and if you’re going to go visit there, there’s really two paths. One is on the consumer side, the customer side, that is, somebody that may want to leverage the platform to gather consumer opinions through voice. And the other is to actually sign up as a panelist to provide feedback. So I want to talk a little bit about your platform. What type of insights are being captured in your voice surveys?


So right now, primarily, we have been focused actually on testing usability testing, mostly for designers and developers, skills and applications. So if you are building an Alexa skill or Google action, and you want to get a gauge on how usable it is, whether people are understanding it, whether one particular approach makes more sense than another, you can use our platform to quickly and easily test with real world users. And we are able to do all of our testing directly on devices. So you can test any Alexa-enabled device. We used to do the test on any Google Assistant-enabled device, and we provide a level of data on those interactions that is exactly unavailable anywhere else in the market today. So it is designed essentially… if you’re building something on voice to get real user feedback and really deep, detailed feedback on exactly how people are using your application, Pulse Labs provides a platform and a panel for providing and gathering that feedback.


So I have not come across a business exactly like yours in our space. Did you do any pivots? Was your start different from where you are right now?


We have not done any. Small pivots? Absolutely. Changes in approaches or changes in focus? I would say absolutely. Major pivots in what our product offering is and what our vision is? No. So our vision from the very beginning has been to provide user research to your real world, real people… user research to anybody (brands, developers, designers, agencies), basically anybody who wants a presence on voice, and wants to understand how real people are using voice, and how real people are interacting with voice, and how they can effectively build their presence there.


In 2023, $80 billion dollars is the projected number that will be spent on voice devices in a voice consumer journey context. What do you think research will look like at that point in time as we see such a migration of the consumers’ expense moved to that environment?


I think that research is going to be based around “How do you make this as easy as possible for the users?”, “How do you make it as convenient as possible?”, so they have easy access whenever and wherever they needed. But also, if you are a brand, “How do you remain top of mind here?” Do essentially, how do you be the orient, and how do you set yourself up so that if a customer just wants to order paper towels or something like that, it’s your paper towels that they are ordering. And that is part of the big play for the voice platforms: they want to have some control and say over who gets that top position. With Google AdWords, it’s always a fight to be on the first page. With voice, it is going to be a fight to be the top, the number one, the one that is recommended, and the one that is provided. There is going to be a lot of research, a lot of understanding devoted to how to make yourself number one, and then how much number one is worth.


Yes, that’s really interesting, especially in the context of how many generic brands are now owned by Amazon and Google. This speaks to the overall importance of ensuring that you are “the Kleenex” of your brand category.


Yes, exactly, exactly.


All right, so the NEXT conference is coming up. You are going to be talking about voice. What is one practical take-away? I know that you are tilting your cards here, but what is one practical take-away that our listeners can gleam from your upcoming talk?


So the practical take-away will would be if you are a marketer or a brand and you want to build something on voice, what you want to do, what you want to focus on are one or two very key use cases that voice can do better than what is currently available right now, that are valuable to you, and then execute on those. Too often, we tend to see either brands think: “Okay, we’re going experiment with this. Let’s put together some application”, and it might be either frequently asked questions application, or maybe they’ll just say: “Let’s take the API that we have, that feeds all of our product line for our website and just connect it to Alexa.” Usually those approaches don’t work so well.

So the important thing is to think of things like what we just talked about, such as the ability to reorder paper towels at the point when you’re done using your current batch, and make that seamless and easy. Those are the sort of approaches that are most successful, and that we will see the best ROI.


Yes, that’s great. I think that the application of Kmart in Australia —I heard this through the Voicebot podcast, which I’m sure you’re listening to, and I’m going to try to distill the information a bit – they were talking about how they actually had a tremendous success. I guess there’s some legislation around not being able to procure a product through voice yet. But the ways that Kmart became dominant in a voice framework is that they provided proximity to the actual product. So if the consumer wants to buy something, they would say: “Is it in stock?”, or “Where is it near me?” and that is how they would get directed specifically to the store. So it is an interesting story for me in that they started talking about how the brand is empowering the consumer and getting close to them, adding the value. Another one, I think it is Chrysler, that has an automatic start feature on one of their automobiles. It’s actually one of the top 100 Alexa’s skills. So it could be cold outside, and you can just tell your voice device: “Hey, start my car.” And it will start. It’ll warm up the car for you before you before you get in. The more the brands start adopting this technology, and the better they’re going to be positioned when this action stuff actually scales.


Yes, exactly. Exactly.


Well, I can’t wait to hear your talk. My guest today has been Dylan Zwick, co-founder and Chief Product Officer of Pulse Labs. Thanks so much for being on the Happy Market Research podcast, Dylan.


Thank you very much for having me. It’s been a pleasure. Thank you. Thanks a lot.


For all of you who are listening, if you’re not signed up for the Insights Association’s NEXT conference, I would highly recommend you do that. Again, that is June 13th and 14th in Chicago. You can also find out information on our website I’ll be including links to Dylan’s information and his company’s information in the show notes. I really hope to see you at the NEXT conference. Have a wonderful rest of your day!