Happy MR Podcast Podcast Series

Josh LaMar, Co-Founder of Authentique UX on Elements of a Good Participant Question

My guest today is Josh LaMar, Principal Researcher and Co-Founder at Authentique UX.

Find Josh Online:

Email: josh@authentiqueux.com 

Web: http://www.authentiqueux.com 

LinkedIn: https://www.linkedin.com/in/joshlamar/

Facebook: https://www.facebook.com/authentiqueux/ 

Find Jamin Online:

Email: jamin@happymr.com 

LinkedIn: www.linkedin.com/in/jaminbrazil

Twitter: www.twitter.com/jaminbrazil 

Find Us Online: 

Twitter: www.twitter.com/happymrxp 

LinkedIn: www.linkedin.com/company/happymarketresearch 

Facebook: www.facebook.com/happymrxp 

Website: www.happymr.com 


“Clap Along” by Auditionauti: https://audionautix.com

This Episode’s Sponsor: 

This episode is brought to you by Lookback. Lookback provides the tools to help UX teams to interact with real users, in real-time, and in real contexts. It’s Lookback’s mission to humanize technology by bridging the gap between end-users and product teams. Lookback’s customers range from one-man teams building web and app experiences to the world’s largest research organizations, collectively ensuring that humanity is at the core of every product decision. For more info, including demos of Lookback’s offering, please visit www.lookback.io


Jamin: Hey everybody, this is Jamin. You’re listening to the Happy Market Research Podcast. My guest today is Josh LaMar. He is live from Brazil. Josh, how are you today?


Josh: Hello. I am doing very well. It’s a nice warm afternoon in Brazil.


Jamin: San Pablo?


Josh: Yes.


Jamin: Specifically?


Josh: São Paulo.


Jamin: So tell us a little bit about yourself. You’re a UX researcher, how in the world did you wind up in that job, and what kind of customers are you working with?


Josh: Well, I didn’t initially set out to be a researcher. And it’s funny because I did my undergraduate degrees in music composition and English poetry, which are seemingly as far as you can get away from research. But then I pivoted a little bit, and then went to grad school in human-centered design and engineering. And I’ve been playing with this idea, I think throughout my career, of the role of creativity in science and how important it is to use creativity to create research methodologies, and to answer questions that we have about our users. It ended up being a really great fit for me.


Jamin: That’s a big chasm between those two things, right?


Josh: Yeah.


Jamin: To say the least. How did you bridge it professionally?


Josh: I started out thinking, what can I do with my English degree? And I started out by getting a certification in technical editing, thinking that I would become an editor. And then afterwards, I found this master’s program at the University of Washington, like I mentioned, human-centered design and engineering. And when I finished the program, I was like, oh, I’m qualified to do research now, and people are way more interesting than commas.


Jamin: That’s true, I guess.


Josh: They’ll always surprise you, and you just have to be open to seeing, what’s gonna happen when you meet this new person? And the act of doing research and going and talking to people, and especially visiting people in their homes, it’s really fascinating for me. And it’s interpersonal and it’s empathetic. And that’s, I think, what draws me and keeps me doing research.


Jamin: The topic for today’s discussion is centered around questions, the anatomy of a question as it relates to research. It’s funny, because as I’ve done this interview now a few times, I’ve realized that my first question is actually a flawed question. It’s hilarious. The type of question that we’re talking about here is really at an interview level, right?


Josh: Correct.


Jamin: So, how you would ask or frame a question in such a way that it would be considered useful to gather information from a participant.


Josh: I think that the way that you frame a question is very, very important because you have to be at the right level. And what I mean by level is that, if you start off an interview by saying like, well, tell me how you check your email on the weekends, you’ve just scoped it so narrow and really, you might be interested in something else. I was the research manager at Outlook for several years, so I can use email as a really easy example of things that I’ve done research on a lot. So it’s really important to start very broad and then move into the specific. And an example of a broad question might be, tell me how you communicate with your friends and family, much more broad than just email. And then as you start getting into it, you’ll find more interesting things. The framing is so important because when you frame too narrowly, you put this box around the user. And the user thinks, I think that they want to hear just this part, and so they only share the things that are in that box. But when you add a broader box from the beginning, then everything else is open. And you might find something that’s even more interesting just by asking a broader question.


Jamin: So it’s thinking of it in a way like a funnel, is how you’re framing it. And so you don’t start with the specifics, you start with the broader. And do you find that helps it-? Let me reframe the question. What are some of the primary benefits of starting broader and then narrowing through discovery?


Josh: I think that the biggest benefit is that you don’t limit the response into the mental model or mindset of the user, or of your own mental model and mindset. As soon as you start limiting yourself, you’re limiting what you’re gonna hear and you’re focusing too narrowly. I think that by broadening it first, you really start to understand the landscape a bit better. And then from there, you’re able to dig into the things that are interesting. And it’s not necessarily the things that are interesting to you, it’s the things that are interesting to the user as well. And that’s what’s more interesting and more important when it comes to doing research. Is, what they think, not what I think and how I compartmentalize all these different ideas together in my brain.


Jamin: How have you framed this out with your customers or your constituents inside of-thinking about like Microsoft?


Josh: Actually, similarly, I think I explain it a lot just like that. Typically, when I’m explaining how I come up with a research methodology, I’ll be talking about the approach, the research approach. And I’ll start broad, like I mentioned, and then narrow in on the things that the team is more interested in. So for example, in doing research on email, it would start with, hey, how do you communicate with people? And then maybe it might go to, where does email fit into the rest of your communication mediums that you use? And then it might go into more specific things that the team is interested in, like testing this particular feature in the email or application itself.


Jamin: That makes a lot of sense. And I can imagine that you’ve had some interesting-like, understanding the context of the usage, thinking about email, probably has been as insightful as some of the initial objectives of the more narrow research.


Josh: Yeah. And I think that another really interesting part about this, when you’re thinking about how you’re framing a question, is that, when you start framing it too narrowly, you can end up priming the user to talk about specific things that they think that you want to hear. Whereas, if you’re starting more broadly, you’re not priming them in any way. You’re starting with like, how do you communicate? In general, and not, what do you think about this particular feature? And that way allows for the unexpected to happen in the interview, which is the most interesting part.


Jamin: So that’s the piece to dive into, right? It’s when the unexpected happens in an interview, that’s the most interesting part. That’s exactly right. Because I’ve done a fair amount of qualitative and quantitative and you sort of, after-I failed, I really failed. I did terrible at research for like three years. And what would happen is, we would do these focus groups around the nation or whatever. And after about three, it kind of felt the same. Like, every focus group started feeling like the same conversation over and over again. My failure in that was I wasn’t starting with this broader context to then narrow it down. Instead, I sort of like developed my assumptions or story and then after that, I basically started framing my questions to self-fulfill this narrative. Sorry everybody that I charged 25 years ago. They did get some value, but you get my point.


Josh: Yeah, I think that that’s what happens too, is you’re essentially telling the user, this is what I think, tell me what I think and in your own way. Or you’re saying, tell me, just reinforce what I already think. And that’s a huge problem as a researcher because then we just start reinforcing our already held beliefs. And like I mentioned earlier, this idea of what’s interesting is what is brand new and what is unexpected. Like, you always know in the back of your mind, well, we want to find out about these five things, or we have this feature that’s gonna do this thing, so we want to understand if people care about this. But then when you’re open to allowing people to share about other related things and you’re not priming them to say, oh, he cares about this feature. It’s more about, well, what do you care about? What are the problems that you have when you’re doing this, in general? There could be problems that we don’t even know about, that if we don’t ask, then we’re spending all this time trying to build this feature that does this thing that people don’t really have that much of a problem with. And so it allows us to really find the latent needs a lot better.


Jamin: It takes a lot of energy, right, when you’re in the field doing the interviews and then trying to find-discover the new or the unknown. It’s an amazing drain.


Josh: It really is.


Jamin: It’s funny, it’s even physically exhausting, which is like-the pre-researcher Jamin, would have had no idea that that was true. So, do you have any tips or ways that you maintain your energy?


Josh: One of the things that I think is really important is to not overload a research schedule. So, yeah, you could do like eight interviews in a day, eight one-hour interviews, right? That’s eight hours. That’s a day of work. But honestly, it’s ten times more work than I would do. And it’s so much better to just pace things out in such a way you are able to have a break to think about nothing, to check social media and decide that you’re going to look at silly cat videos. Things like that are actually really good for your brain because your brain needs a break between all of this really intense thinking that you’re doing. I think that junior researchers are often surprised by how difficult it is. For many years, I was coaching a usability study course at the University of Washington and my former grad program. And I’d be there in the lab with them for the first time that they’ve ever done a usability session ever, ever. And we’d talk about it afterwards and they’d always say, oh, wow, that was so much harder than I thought it was gonna be.


Jamin: Right.


Josh: You can make all the mistakes you want because it doesn’t really matter, this is just like practice. But they always are surprised at how difficult it really is, because you do have to be on top of so many things all at once. You have to know what the goal of the research is. You have to know about the different features that you’re trying to test, what decisions the team is trying to make in order to-like, this research has to inform these business decisions. You have to know what those are and then follow up on them and ask around them, as to my other favorite technique, ask around things, but don’t ask them directly. And so there’s this back and forth that goes on where you’re like, well, what do you think about this? And, tell me about how this works for you or not. And you have to kind of like dance around the topic until at some point, if they just never get it or never think about it, then you can ask it directly at the end. But there’s this dance that you’re dancing around the, what matters to you, without telling them what you want to hear about.


Jamin: That is so hard to do because it feels like the wrong question when you’re in the moment and having that. And there’s such a risk-a perception of risk, I should say, because your client may be present or may look at the transcripts and they care about the answer to their business question. Not-thinking about Outlook, not broader communication themes that may involve things like, in the old days, mail or what have you, or faxes. I’m definitely dating myself. So the risk there is that-or the perception of the risk, I should say, is that, gosh, am I gonna be able to meet the objectives in the allotted time that I have to meet with a participant? But really the risk is, by not doing that, it’s much greater, because you ultimately may not get to the undiscovered truth.


Josh: I think that over time, I’ve realized that I trust the system, I trust the process. That when you create this opportunity for people to talk about what’s interesting to them, they will start doing it. And if they go off-topic, a little off-topic is fine and you can always bring them back. My favorite trick for that is to ask a yes or no question, and then take control of the conversation and ask about something different.


Jamin: Oh, that’s so clever. Thank you for that hack.


Josh: It’s like, oh, do you do that every day? Yeah, I do. Great. Let’s shift gears for a minute, and I want to talk about this other area now.


Jamin: That is the quote of the episode right there, that is gold.


Josh: It is. Just even understanding like, this is an open-ended question, this is a close-ended question. I’m gonna use them very intentionally. So I’m gonna ask an open-ended question when I want to just hear you talk about anything that you have to say about whatever topic it is. And then a closed-ended question could be used to close things off and say, yes, no, that’s it. And then you take control, shift gears, and then you’re on to the next topic. There’s also a really interesting moment when you’ve done several interviews in a row, I think it’s around five interviews. If you did like a usability test of ten people or an interview with ten people, around five or six people, you know pretty much what they’re gonna say. It starts off with like, the first few are, oh, yeah, that’s really interesting. Around three and four, you’re like, these are some themes that are coming out. Around five to seven, you’re like, this is exactly what you’re gonna say. And then by seven through ten, you know exactly what they’re gonna say. And there is this moment where they say enough and you’re like, you said the exact same thing, we can move on. And so that’s why the final interviews always go so much faster, because you can already hear the same thing and you’ve heard them say everything that everybody else said.


Jamin: It is uncanny. And then the temptation there is, for me anyway, is to treat that as a quant study because you feel like you’ve definitively answered for that particular segment, right?


Josh: Yeah, that external validity is really important to iterate.


Jamin: For sure. We’ve really covered this already, so common mistakes that you’ve seen in framing questions for an interview?


Josh: Yeah. I think that the biggest mistake is to either ask a leading question or to frame it too narrowly first. We’ve talked about framing narrowly first, so I guess we could talk about leading questions now. Which are things like, tell me how amazing this product is. That’s an over-exaggeration, but it can be much more subtle too. Like, if you’re only asking about the positive aspects of something or you’re saying, oh, this is a really great feature, isn’t it? Well, what did you just do there? You told the user-you primed them, number one, to say, I like this feature, and then I created this tag question like, isn’t it? Don’t you agree with me? You should agree with me because I’m the smart one here. You just made the user feel dumb, and then you also told them exactly what you want to hear. So what are they gonna do? They’re gonna tell you what you want to hear because they want to make you happy. And it’s so important as a researcher, to be very neutral and to ensure that you’re not letting too much of your own feelings ever come out. Because as soon as you start letting on like, this is really dumb, isn’t it? Yeah, I don’t really use this, but we need to test this for our client. Can you just tell us that thing? You’re throwing out the whole study data if you do that because it’s too leading, you don’t want to lead them on to the answer. The answer is what they think, not what you think.


Jamin: You’ve seen a lot of questions, what is one of the worst?


Josh: So tough. It’s gonna be anything where it’s too, too leading like that. Like, oh, this is great, isn’t it? I used to coach PMs and designers at Microsoft on my former team on how to talk to users, and I have a whole presentation that I would give on how to talk and not talk to users. Because really it’s about listening, it’s not about talking. And whenever I’ve seen these examples, I will call it out after the session and say, oh, hey, when you said this, it was pretty leading. So the best thing that you can do, if you find yourself asking a leading question, is to ask the opposite next. So if you said like, oh, tell me how great this is, you could say, is there anything that isn’t as great about this? Or, how was this difficult to use? The better thing would be to say, how easy or difficult was this to use? Or, tell me about your experience using this thing. Those are much more neutral responses and questions. So you can try to be as neutral as possible, but then if you ever find yourself being a little bit more leading, just ask about the opposite end of the scale. It’s really all about being neutral, that’s so important.


Jamin: That’s probably, for me, the biggest energy drain in interviews because I like people, fundamentally. I just like people and I want to connect with them, so it’s like-I don’t know, sort of like innate desire for me. I’m sure it has a lot more counseling before I get to the root cause of that. And so one of my big energy drains in interviews is maintaining neutrality as opposed to building a relationship or some level of rapport.


Josh: I think it’s really important to develop just enough rapport in order to have that person feel comfortable with talking to you. And you’ll have a little back and forth or chit chat about the weather from the elevator to your lab or something like that, it’s totally fine. You want them to be comfortable in sharing their deepest, darkest secrets with you. And if they don’t feel comfortable with that, there’s no way they’re gonna share anything that really matters to them. But I think that it’s also important to keep the focus on that other person. Even if you’re thinking, oh my god, me too. I feel like that all the time. You, uh-huh, tell me more about that. Neutral, and you can think it without sharing that. And as a really empathetic person, I feel that same thing that you just talked about, where I really want to connect with the person because I feel their pain. And I feel like that’s one of the best things that a researcher can be is just empathetic with people’s pain. Because it’s not easy, using technology is not easy and we try to make it easier, and that’s really important. And then when you see people, you see them struggle, usually the worst is in a usability study where you see like, you know how the team decided to build the feature, and you know why they didn’t put the button in the place that the user is looking for the button, and then you just see them go in circles. I’ve seen people go in circles looking for things and never find it. And it’s like, it’s sad, but this is the kind of thing that the team needs to see in order to understand, oh, hey, this is really confusing. And then after the session is over, there’s always that moment after you leave, after the cameras are off and you’re like, thank you so much. I felt the same way about these things that you talked about too. And then you can have that human moment when it’s all over, when it’s not thwarting you.


Jamin: That’s so funny that you say that. It’s almost like there’s delayed gratification and that we just need to make sure that we install as a discipline. And you’re right, that level of rapport that you build at the beginning is critical because you need to be able to get to the deep stuff. But you have to refrain from congratulatory responses to answers that they give, because all of a sudden you start creating that feedback loop of fishing, I guess.


Josh: And I’ve seen that thing where-in a usability study, I’ve seen other researchers say like, oh, yeah, that was very good. Great, next, and they’re always too positive and they start getting to this level of congratulating the user for their feedback. Or when they say something negative, always saying, oh, thank you so much, that was so helpful. It’s maybe a thing that we’re thinking, but you have to tone it down a little bit like, thank you for your feedback. Very different than, oh my gosh. And I find that I have to tone down my exaggeration quite a bit.


Jamin: That’s why being a podcast host is way more fun, that’s why I can be. So, you’re a UX consultant, right?


Josh: Yes.


Jamin: If somebody has a UX job, what type of jobs do you usually take? What’s the ideal customer look like, and how can they get in contact with you?


Josh: Well, we just started my UX research company with my husband, we are Authentique UX. You can just add a-Authentique is the conglomeration of authentic plus boutique. Plus, it is the word authentic in French, and so that’s where it came from. authentiqueux.com is our website. To get in touch with us, my email is josh@authentiqueux.com. That’s A-U-T-H-E-N-T-I-Q-U-E-U-X.com. And yeah, we do research, UX research. We’re based in Paris, and so we can easily do stuff in Paris. But my husband is from Brazil, and so we’ve done research in Brazil as well. And yeah, we’re pretty open in terms of methodologies, but we’re really looking at trying to find the interesting depth of whatever question it is that we come across, because there’s always more there. And we have a team of people actually that are-they all have over ten years of experience. And so we’re really looking at, how can we do these things very, very well and answer those deeper needs that people have and also address the business questions?


Jamin: My guest today has been Josh LaMar. Josh, thank you so much for joining me on the Happy Market Research Podcast.

Josh: Thank you. It’s been great to be here.


Jamin: Everybody else, if you please take time, screen capture, share, tag us, LinkedIn, Twitter.