Happy MR Podcast Podcast Series

Ep. 515 – Lisa Wilding-Brown, Chief Executive Officer of InnovateMR, on the Characteristics of a Fraudulent Respondent and how to Filter Them Through Survey Design

Our guest today is Lisa Wilding-Brown, Chief Executive Officer of InnovateMR.  

InnovateMR is an independent sampling and ResTech company that delivers Faster Answers™ from business and consumer audiences utilizing technologies to support agile research.

Lisa is a veteran of the marketing research industry and has served as an executive at uSamp.

Find Lisa Online:

Find Jamin Online:

Find Us Online: 


This Episode is Sponsored by:

The Michigan State University’s Master of Science in Marketing Research Program delivers the #1 ranked insights and analytics graduate degree in three formats: 

  • Full-time on campus 
  • Full-time online 
  • Part-time online

NEW FOR 2022: 

If you can’t commit to their full degree program, simply begin with one of their 3-course certificates: Insights Design or Insights Analysis. 

In addition to the certification, all the courses you complete will build toward your graduation.

If you are looking to achieve your full potential, check out MSMU’s programs at: broad.msu.edu/marketing

HubUX is a research operation platform for private panel management, qualitative automation including video audition questions, and surveys. 

For a limited time, user seats are free. If you’d like to learn more or create your own account, visit hubux.com.


Chueyee Yang: Today is March 21, 2022. Happy Monday. This is Chueyee Yang, the show’s producer, and you’re listening to the Happy Market Research Podcast. We have a very special guest today, but before we get to that, here’s a word from our sponsors.


Jamin Brazil: Support for the Happy Market Research Podcast and the following message comes from Michigan State’s marketing research program and HubUX. The Michigan State University’s Master of Science in marketing research program delivers the number one ranked insights and analytics degree in three formats: full-time on campus, full-time online, and part-time online. New for 2022, if you can’t commit to their full degree program, simply begin with one of their three course certifications, insights design or insights analysis. In addition to the certification, all the courses you complete will build towards your graduation. If you’re looking to achieve your full potential, check out MSU’s programs at broad.msu.edu/marketing. Again, broad.msu.edu/marketing. HubUX is a research operations platform for private panel management, qualitative automation including video audition questions, and surveys. For a limited time, user seats are free. If you’d like to learn more or create your own account, visit hubux.com.


Chueyee Yang: This is episode 515, and in less than a month, the South Korean K-pop group BTS will be starting their four-day concert at the Allegiant Stadium in Las Vegas. Their most recent release was back in October when they hit the Billboard Hot 100 and number one with the song “My Universe,” which is also a collaboration with Coldplay.




Jamin Brazil: Hi, I’m Jamin. You’re listening to the Happy Market Research Podcast. Our guest today is Lisa Wilding-Brown, chief executive officer at InnovateMR. InnovateMR is an independent sampling and ResTech company that delivers faster answers from business and consumer audiences, utilizing technology to support agile research. Lisa is a veteran of the industry. She started her career at Harris Interactive and was an executive at uSamp. Lisa, thank you for joining me on the Happy Market Research Podcast today.


Lisa Wilding-Brown: Thank you, Jamin. It’s great to be here. Thanks for having me on your show.


Jamin Brazil: It is an honor as always to chat with you. You’ve had tremendous success and just a rocket ship career. Let’s start with some context. What did your parents do and how did that inform what you do today?


Lisa Wilding-Brown: Oh, that’s a great question. So I grew up in western New York, in Rochester, New York. Spent most of my childhood here in New York. And my mom was an at-home mom for most of my childhood, although she went into real estate when I was about maybe 15 or 16 and became an absolute powerhouse saleswoman in the greater Rochester area for her real estate agency. And my dad was an auto mechanic. So he immigrated to the US from the UK when he was a young man and very spontaneous type of gentleman. When he got to the port of New York, he pulled out a map, closed his eyes, and landed on Buffalo, New York. He went there, couldn’t find a job, and went to the next closest city, which is Rochester, and the rest is history.


Jamin Brazil: Did he really do that?


Lisa Wilding-Brown: He really did. Yeah, he really did.


Jamin Brazil: Wow.


Lisa Wilding-Brown: Yeah, he sailed over on the Queen Mary. And it’s like the stuff that movies are made of, in a way. He grew up in northern England in a really poor family. Was born in the middle of World War II during an air raid. Again, true story. My grandmother had a football helmet on her head as she was giving birth to my dad in her bed and my grandfather was away at the war. So some of those World War II movies that we all have seen in our past lives, they really I think capture some of my dad’s childhood experience for sure. And it’s definitely informed who I am, his story of struggle and how he grew up, and the same for my mother. She was a farmer’s daughter, so definitely a long legacy of hardworking people.


Jamin Brazil: I hear a lot from leaders in the industry about their parents and oftentimes their parents are immigrants into the US. And my guests, they wind up having these extraordinary careers. What do you attribute that to?


Lisa Wilding-Brown: Well, I think if you think about the United States, it’s really this amazing country with a long pedigree of immigration, of people coming from other destinations around the world to start a new life and to follow that yellow brick road to glory, you know? And I think you have to be a certain type of person to risk everything and to leave your homeland for such an uncertain destination where you don’t really know what you’re gonna do, where you’re gonna live, who you’re gonna be with, and what that world looks like. So I think there’s a level of moxie and just brute kind of just risk, just going for it. A certain type of person fits that profile, and my dad definitely I think fits that profile. And I also married a foreigner too, so my husband’s from Dublin, Ireland. That’s where I met him when I was living abroad for several years. And we made a decision to come to the US and Alan said, yeah, I wanna go for it. And so he’s an entrepreneur, my husband is. He started his own business here. And so it’s true, Jamin. I think a lot of people that come from other countries and come to the United States to start a new life really have this I guess kinda X factor about them, and that later informs how future generations within their family tree operate, I guess.


Jamin Brazil: Maybe part of it is the vision that is required to make that cultural transition is fairly unique and you’re really quite literally just inventing your life as opposed to being born into it with a set of assumed rights of achievement and that probably just carries over. And it’s also interesting the point that you made about sort of that grit and determination and willingness to put the time in in order to achieve success, and a lot of that probably is just a function of the lack of entitlement. No offense meant to myself or anybody else who’s not an immigrant or child of an immigrant, but it definitely is a trend that I’ve seen. So I wanna move into really our core discussion. 2021 is over. Some people are really excited about that. What are three things that you’re grateful for coming out of 2021?


Lisa Wilding-Brown: Oh, that’s a great question. ’21 was a wonderful year for me personally and professionally. I’m super grateful for the health of my family and the great health that I enjoy. I think your health is your wealth, is what I like to say, and so I’m forever grateful to have good health and to have a healthy family. Speaking of family, I’m super grateful for them. It’s probably not very easy to be married to me at different times, so I’m really grateful for my husband and his constant support. I obviously had a big change in my career this last year being appointed to CEO here at Innovate and my family has been so supportive through that transition. I’ve always been a bit of a road warrior, so I did get on the road for a few different conferences last year and that was sort of a strange dynamic given that we really didn’t go anywhere in 2020. But my family has always been so, so supportive, and my mom as well. She is my biggest fan. She listens to every podcast I’m on, so hi, Mom. I’m sure she’s gonna be listening to this one too. She’s been a wonderful support system because my husband travels a fair amount for his work as well and it’s just really great to have that extended family support for two sons. And then the third one is just, I would say just this incredible sense of opportunity on the horizon. I think 2020 was such a scary year for so many people both personally and professionally. We didn’t really know how things were gonna net out. And then I think everyone sorta said, right, OK, it’s time to pull our bootstraps up. Let’s get serious. Let’s figure out what this new normal looks like and get on with life. And I think that is something that I really observed not only within my own family, but the countless friends and other industry veterans that I have the privilege of knowing and working with. So it’s just really great to see that there always a need to measure and there’s an insatiable appetite for data now more than we’ve ever seen. And so that’s a really, I think a wonderful thing that emerged during the last year, was that everything’s gonna be OK. Life will go on.


Jamin Brazil: Yeah, and it does go on. Some people have had challenges on a variety, inside of my network, a variety of fronts, and all the things that you’re articulating are exactly right. It is, quite literally health is wealth, and it’s oftentimes the case that we forget that when we’re in a healthy state. But are very quick to remember it and constantly being reminded about it as friends and family and even individually we have exposure to COVID or other things. But it’s certainly redefined, and this is categorical, how we live our lives. So as you reframe 2022, what do you see as our industry’s biggest challenges?


Lisa Wilding-Brown: Hm. Well, I think there’s just a ton of opportunity, which is incredibly exciting, but in terms of challenges, I think there is a strong appetite to leverage technology. And that’s not necessarily a new thing, Jamin. Especially where I sit in the industry in the sample space, I think as a sector within the industry we’ve always really embraced technology, programmatic strategies to be smarter and more efficient in the way that we deliver respondent audiences across the world. But I think there is this notion of different companies running really fast and hard at technology and then there are others out there that maybe don’t have that within their resources or capabilities to run quite as hard and quite as fast. And so I think that is challenging for companies that fit that particular profile. And then I would say the other challenge is just around quality. Sample quality is so critical. It is the oil that makes this machine run. Without survey participants taking surveys, and good, high quality ones at that, everything we’re trying to accomplish as an industry can be called into question and the data can yield inaccurate results. And so cyber fraud has continued to be a challenge, not only for our industry but anyone that does anything online. And so I’m just really looking forward to seeing what new, emerging companies come out with new solutions that can be leveraged by our space. I think our work is never done on the cyber fraud front. In fact, it’s just getting worse and worse year over year. We’ve seen that. Our friends over at CleanID partnered with me when I did a speaking opportunity for the Insights Association town hall last spring and I worked with them to get some data, just what was the whole entire ecosystem seeming across all of their different clients in their portfolio. And they’ve said that they’ve seen over a 200% increase in survey fraud since the start of the pandemic. And so it really is a testament to the challenging times we face and the need to not only indoctrinate both methodological as well as technological strategies to get in front of that the best way we can. But cyber fraud’s not going away, and anyone that tells you it is not telling the truth because it’s getting more and more sophisticated as each year passes, I find.


Jamin Brazil: I did a presentation at SMR, or for SMR, excuse me, globally in December and it was actually on this very topic. I had had a customer that– So my company, we recruit everything on social media as opposed to through sample providers, which is a really interesting framework. But we wound up having a disproportionate amount of fraudulent participants participate in this research. And I was really surprised because I wasn’t in the sample marketplace because I’m sourcing things on Facebook. And it was actually 18.5% were fraudulent participants. They were saying– And we’re paying a reasonable incentive, et cetera. So that was really interesting, that fraud exists categorically. You have the other framework in the US where there was over $100 billion that was fraudulently claimed from the US relief act, financial relief act. And again, so you see fraud is part and parcel with our digital economy. But probably the thing that really disturbs me the most is the fact that I did a study, an example, just a test study really, to see how a low IR project would perform on the sample marketplaces, leading sample marketplace. And I was– So I created a survey with screening requirements of a CEO of a Fortune 5,000 company. And I was able to source 100 completes within just a day for under $3. Now–exactly–you and I both get the punchline there. Whereas a lot of people would maybe turn a blind eye and say, woohoo, we have 100 completes. But it’s all bullshit, right? There wasn’t one real complete in that whole sample frame.


Lisa Wilding-Brown: Mm-hmm, mm-hmm. It’s scary out there. The Internet is a messy place. And I think it starts with really having conversations like this one, Jamin, where you’ve really– This industry needs transparency. We need to talk about what the problems are, what kind of issues we’re seeing as an industry. Listen, there’s no bad source versus good source. Quality really exists at the respondent level, and that’s the way we need to approach it. And you can’t just sorta say, oh, well, I’ve got a digital fingerprint, I’m all set. No. Because people can spoof their IP address. They can take multiple surveys from different devices and work in a more collaborative model, which we’ve seen a lot of kind of rise of survey farms. There’s all different permutations. You think of, I always say this, you think of quality exists really on a spectrum. On one end you have those very nefarious, very malicious people that are doing this at a professional scale to really earn those incentives in the aggregate and it really does add up. One survey might offer a dollar, but if you are able to find a way to take that survey multiple times, that really does add up and it becomes very material and meaningful in terms of currency, especially for countries that are feeling economic duress and developing countries. But that’s not to say that there’s not a ton of fraud coming out of the US, because there is too. And so I think it’s really about recognizing that, yeah, there’s those extreme cases, but then you’ve got real people in the middle that maybe are just overzealous in their responses and aren’t there for the right reasons or willing to provide thoughtful responses, and we need to address those folks too. And then you just have natural duplication that can exist because you’re multi-sourcing with different vendors. So there’s different permutations and there’s really different strategies as a business and as a researcher that you need to employ to filter out the noise. And listen, there’s dissonance in data. That’s half the fun of it, in all honesty. There’s always going to be noise in data and I love to sort of put the researcher hat on and find and use different methodologies to really catch the unsavory or catch the undesirable behaviors that any survey instrument will invite just by the nature of what it is, of offering incentive to provide responses to a survey instrument. You’re going to get great people to take that survey and then you’re going to get people that are just not taking it as seriously as you are, and you’ve got to take a proactive approach and really get the power back. And that starts by asking the tough questions and using sources that have the appropriate level of validation, but never taking your foot off the pedal. You have to assume that you’re going to have some noise in your data and really proactively include elements within your survey design to be able to diagnose those cases when they come through.


Jamin Brazil: Yeah, but it’s to degrees, right? So if you have a 35% fraud rate, the problem is that you’re not actually able to get rid of or cull the full fraud participant inside of your sample frame because it’s just too much of the actual sample. The other problem that you mentioned, and I think it’s exactly right on one hand but maybe not right on the other, which is the sourcing. So a lot of sample is sourced through paywalls, as you know. And in a paywall framework, I’m playing virtual poker and I wanna refill my kitty because I just lost to Lisa who’s a much better poker player than me, so I have to go do these things. And so my motivation there is quite literally just to refill my hand as opposed to give a thoughtful response. And that kind of starts begging the question of, well, what is the right incentive framework? Because if we offer an adequate incentive, then we know, you and I both know because we do this research on a regular basis, participants want fair trade. Unless they’re in a private community where they really care about the product or the service, they just care about fair trade. And now all of a sudden you’ve got average CPI for a B to C survey is sub-$1 in the marketplaces.


Lisa Wilding-Brown: Yeah. That’s just, you set that up, that test up with serious intention. You’re looking for the fraud. If you’re expecting to get CEOs at that price point, those very specific audiences like C-level executives, they’re typically not in marketplaces, right? You have to go elsewhere to find those folks. And I think you touch on something that’s really interesting around incentivization. So incentivized traffic, you have to be really careful when you’re setting up recruitment campaigns around those very generous incentive models, because when you dangle that carrot to yield instant gratification, you’re definitely going to increase the proclivity for or increase your risk for nefarious people with bad intentions. That’s why I’ve always really appreciated a more threshold-based model, meaning here’s what you’re going to get in the future granted that you participate in a meaningful and thoughtful way. So more of that kind of banking concept and sort of an exchange. We’re making this exchange and having this transaction with a panelist or survey participant that you do X, you get Y. But that Y I think is really critical, and where that Y is introduced in the user workflow is really critical to consider too because it can invite very bad behavior. So I agree with you on that for sure.


Jamin Brazil: A practical example of a mistake that was made in a project I recently managed, it was a private community, a $25 incentive was the payment, which is very generous.


Lisa Wilding-Brown: Mm-hmm.


Jamin Brazil: But it’s a hard-to-reach, low-IR group, so there’s a lot of value in having them. It was filling about 20 people a week and it was a hard 20 people and then all of a sudden we were getting about 50 to 60 a day and so everybody seemed really excited about that. But when we pulled it back, what was happening is it was somebody had figured out the path through the survey or the screener and they were basically just passing around the link, getting other people to fill it out in order to game the system. So now, the client has shut down the incentive component, to your point, and it’s exactly the right answer. And it’s a human governed approval process that is triangulated, validated based on a variety of different answer profile questions like, are they the same person that they said they were on social media? And can we validate that in two different areas? Are they actually employed on LinkedIn at that company they specified? Et cetera, et cetera. So yeah, but it’s a lot, to your point, it’s a lot more work.


Lisa Wilding-Brown: Mm-hmm, mm-hmm. I mean, that’s why I think it’s important to ask those questions, right? When you’re interviewing a new sample provider, you’re looking at a sample marketplace or working with a different provider, ask those questions. ESOMAR recently updated last year, as you know, the ESOMAR 28. Now I think it’s deemed the Online Buyer Guide. But it’s a wonderful, wonderfully rich resource to really help clients and buyers of research, buyers of sample specifically, what are the types of questions I should be asking my suppliers to really understand how they’re sourcing? Because while I do believe quality exists on the respondent level and you can have a good panelist or a good survey respondent sitting right next to a bad one in the queue, there are definite themes or outcomes that we know to be true when certain dynamics are in place. Like that really very high incentive offering for low incidence population, the reward is so high, people will, especially if you’re using like a public URL so not a password protected URL, that would make you very, very vulnerable because it can be so easily shared. But there’s no silver bullet here, Jamin, I think is the thing to say, is there’s no silver bullet, it’s never going away. It’s really about being very smart and thoughtful on who you’re working with, asking the questions. Having that transparent dialogue is super, super important. But then again, never assume anything. Because what I’ve seen in my testing in my years of experience is I’ve worked with people that recruit into our panel and they have a great pedigree, a longstanding record of great performance, and then all of a sudden one day it changes. Because remember, we go back to that notion that the Internet is a very dynamic and messy place, and your partners can be making changes further upstream that you’re not privy to. And when that happens, not only you as a sample provider are caught off guard, but then any client that’s using panelists from that source can be caught off guard. And so that’s why you have the sort of interdependency that exists across the supply chain and it really requires open, transparent conversation, ongoing testing, which really serves as the litmus test because again, it occurs at the respondent level. So even if you were to do a test of 500 or 1,000 respondents from a new recruitment partner, they could artificially create some dynamics to make their sample look really great. And then when they think the coast is clear, they sorta change for a cheaper model. That stuff happens in our space all the time. And so it’s really important I think to hold hands with the folks that are being open and transparent, recognize that cyber fraud is not going away, and really arm yourself with some protections in the survey. But to your earlier point around the sample frame, it is really key. You don’t wanna have to clean out a high percentage of your sample. So it is really about working with great partners that have that transparency and that openness and are really partners versus vendors and really create that dialogue so that you can get smarter. Because the good news is there is a lot of different things that you can do even before the survey experience to just ensure that you’re getting better quality. We have incredibly low scrub rates at Innovate and that’s by design because we’ve really indoctrinated a lot of different mechanisms in our registration path, in our pre-survey path, in our post-survey evaluation. It’s really about maintaining good hygiene across your panel or your sample source, whatever it is you may be using to cull out those bad actors and be able to diagnose them when they try to enter your ecosystem.


Jamin Brazil: Audience, this is the most important question we ask ourselves moving into 2022. You need to be paying attention to your sample. Sourcing is a bigger issue than it’s ever been before. We’re entering into an election year. It’s just vital that we are actively managing this and setting expectations of, I’m gonna source a low IR for just a few dollars in a tight timeframe, is just very unreasonable. So things need to pass this just basic common sense framework, and if it doesn’t in your mind, then you really need to draw into question and be active with your sample providers to make sure that they have the right protocols in place to make sure that things are being managed appropriately. All right. So that’s the monologue. We could talk about this forever. Probably everybody else is already bored. So you’ve had a lot of success, Lisa, a tremendous amount of success. What is one piece of advice that you would give someone who’s starting their career in market research?


Lisa Wilding-Brown: Oh, there’s so much. I wish I could go back to Lisa from the early 2000s and give her some advice, but that’s not possible. I would say just stay curious. A lot of people will be like, oh, where did you learn all about cyber fraud? I taught myself, you know? I went to a lot of online webinars. I’ve done a lot of continuing education courses. I’ve liaised with different thought leaders in the industry. You have to really take your career into your own hands. Just like those immigrants coming to the US, like my father, you are the author of your own destiny. And I think when you start to look at your future in that framework, it really helps you to feel more empowered, more in control, and less of a victim. So I would just say stay curious and be in control and lay out that vision for what you have in mind for your future. I think that’s really important and very empowering for sure.


Jamin Brazil: What is your personal motto?


Lisa Wilding-Brown: God, I have a few, but a big one that I learned early in my career is never assume. Don’t assume, always measure. What’s that saying, assuming makes an ass of u and me?


Jamin Brazil: Mm-hmm.


Lisa Wilding-Brown: Have you ever heard that, Jamin?


Jamin Brazil: I have.


Lisa Wilding-Brown: And I think it’s very true. And it’s interesting, I know a lot of people in our space, we’re in the insights space, right? Data is running through and coursing through our veins. We love data. We wouldn’t be in this industry if we didn’t appreciate and love data. But there’s a lot of decisions that get made day in and day out by people that work in this field that are not measuring. They are not relying on data to inform their decisions. They’re making really dangerous assumptions about, it could be about anything. About their business, about their career, about what’s happening in the industry. But I think never assume, always measure is something that is really important to think about. And we all fall victim to it at certain times in our career where we’ll make a dangerous inference or dangerous assumption that has a negative consequence down the line. And learn from that and certainly don’t be too risk averse. We have to take risks to be successful, I think. But you wanna make a decision that is pulling you out of your comfort zone, but you’ve done your homework, you’ve done the legwork to really be smart about the choices that you’re making. So don’t assume, it’s dangerous, would be a good motto for me.


Jamin Brazil: Good advice from a leading CEO in the insights space. Our guest today has been Lisa Wilding-Brown, chief executive officer of InnovateMR. Lisa, thank you for joining me on the Happy Market Research Podcast.


Lisa Wilding-Brown: Thanks for having me. We’ll see you soon, Jamin. Take care.


Jamin Brazil: Everybody else, I hope you enjoyed this episode. As always, screen capture, share this on social. Tag me or Lisa and I will send you a free t-shirt. Have a good rest of your day.