Ep. 312 — Laura Levy & Emma Varjo on the Rise of Esports and how Consumer Insights can Help

This episode is in collaboration with IIEX’s podcast series.

Our guests today are Laura Levy and Emma Varjo.

Laura is a human factors psychologist, specializing in how people interact and engage with technology.

She works at the Institute for People and Technology and she is Research Director of Gaming and Esports Applied Research at Georgia Tech, where she specializes in esports research, games user research, AR/VR, and human-computer interactions.

Laura received her BS in Zoology from the University of Florida, a MS in Biology and a MS in Psychology from Georgia Tech and is expected to earn her PhD in Psychology this Spring 2021 from Georgia Tech.

Emma Varjo is the UX Lead for Frozenbyte Oy.

Frozenbyte was founded in 2001 and headquartered in Helsinki, Finland. Now, with over 110 employees, Frozenbyte has 11 published titles. Most recently:

Boreal Blade which is a team-based melee fighting game with a focus on player vs player combat,
The Trine series which is a best-selling game in the adventure genre
And, Frozenbyte is scheduled to launch Starbase, a space MMO with a fully destructible and infinitely expanding universe, focused on building and designing spaceships and stations, exploration, resource gathering, crafting, trading, and combat.

Prior to joining Frozenbyte, Emma has served as both a software developer and software designer.

Find Laura Online:

Website: www.lauralevy.science

LinkedIn: https://www.linkedin.com/in/lauramlevy

Twitter: https://twitter.com/sciencelaura

Find Emma Online:

Website: https://www.frozenbyte.com

LinkedIn: https://www.linkedin.com/in/emmavarjo

Twitter: https://twitter.com/eevarjo

Find Jamin Online:

Email: jamin@happymr.com

LinkedIn: www.linkedin.com/in/jaminbrazil

Twitter: www.twitter.com/jaminbrazil

Find Us Online:

Twitter: www.twitter.com/happymrxp

LinkedIn: www.linkedin.com/company/happymarketresearch

Facebook: www.facebook.com/happymrxp

Website: www.happymr.com

Watch this episode: https://www.youtube.com/watch?v=L5FqExXM4rg

Music:

“Clap Along” by Auditionauti: https://audionautix.com

[00:00:00]
Jamin Brazil: Hi. I’m Jamin. You’re listening to the Happy Market Research Podcast. Actually you’re seeing the Happy Market Research Podcast if you’re joining us live today at the IIEX event. Today the show is being done in conjunction with IIEX’s podcast series and it is also going to be produced in our regular show feed wherever it is that you consume podcasts. The topic for today is the rise of Esports and how consumer insights are being used to make decisions. This is something I am really excited about as I’m a life-long gamer; been doing it since I was gosh, early days of Apple 2, Atari 2600 and now I’m 49-years-old still gaming with my kids, which is maybe a little bit embarrassing. I think I’m going to probably carry it with me to the grave. We’ve got two guests today that are joining us. Our first one is Laura Levy. Laura is a human factor’s psychologist specializing in how people interact and engage with technology. She started at the Institute for People and Technology and is a research director of Gaming and Esports Applied Research at Georgia Tech where she specializes in Esports research as you’ve probably already gathered, games, user research, AR, VR, and human computer interactions. Laura received her B. S. in zoology, which is super-interesting from the University of Florida, an M. S. in biology, and an M. S. in psychology from Georgia Tech and is expected to earn her PhD I think in six months, spring of 2021 from Georgia Tech. Laura, welcome to the Happy Market Research podcast turned blog.

[00:01:51]
Laura Levy: Thank you very much.

[00:01:53]
Jamin Brazil: Before we introduce our next guest Emma, who is the UX lead for a large video game company out of Finland I’d like to set some context. Maybe you could tell us Laura a little bit about your parents and what they do and how that’s impacted who you are today.

[00:02:09]
Laura Levy: My parents – I come from a family of scientists really and I think that obviously had a lot of impacts on what I do now. So on my dad’s side, he’s a geologist. My mom side, they’re all self-taught naturalists pretty much. And I grew up on Florida where you have all different kinds of ecosystems. So it was very common for the house bookshelves just to be full of field guides and not just this is a bird field guide. It’s like this book is only about wading birds or raptors. So that explains the B. S. in zoology because you had – at least at the time you either had to pick animals or plants. So you were either a botanist or a zoologist and it just seemed very natural that you go for a walk and it would take you an hour to go a mile because you’re like what kind of plant is this, what kind of rock is this. So that had a huge impact on me developing as a scientist up until this point.

[00:03:04]
Jamin Brazil: So your parents, scientists. Were you an only child?

[00:03:09]
Laura Levy: Yes. And in fact, they’re big sailors and big water people so we basically lived on a boat for a lot of my childhood, which made it super-fun because it was like those field guides were in the boat and then you could just go outside or hop in the water and I had a little underwater slate. So my parents also big scuba divers. My dad was – is – well not anymore but a cave dive instructor; my mom, an advanced diver. So I learned to scuba dive before it’s even legal to teach people to scuba dive, which is generally the age of 12. And I had a little slate so I’d hop in and I would be like I saw five squirrel fish and two plain parrot fish and I would do my own logging, be like this reef seems healthy. I’m like ten. But I made that decision.

[00:03:59]
Jamin Brazil: It’s interesting you came from a very tactile, real-world – what I call real-world the physical space that we occupy, and now you’ve kind of moved your career from that to a digital context, to a digital world where actually spending lots of time. Why did you make that transition?

[00:04:16]
Laura Levy: Yeah, that’s a really interesting question because to me the thread of my career makes a lot of sense but sometimes people are like why do you have all these degrees in biology but now you’re working in tech. I studied behavior so I have about ten years’ experience working as a marine biologist, I studied dolphin cognition, I worked for the International Shark Attack Pilot in the US and shark attack behavior. So it was all about like what do animals do in the natural environment and what are the techniques that you can use to understand maybe why they’re doing those things. So when I made the switch, which was honestly around the recession. I worked for a museum exhibition company traveling the Body’s exhibition and the Titanic exhibition, recession happened, and I just kind of worked my network and they needed a behavioral person to look at thousands of hours of video of older adults playing video games. And they needed a statistician. So that was kind of that STEM toolkit that I had. I was like yeah, I could do statistics. And I applied the biology observation techniques to these videos and that’s how I got into Games Users Research. So it’s basically still like what does this organism do when given this thing. In this case it’s a game. We also work on apps and other types of health technologies where we – to me it’s the same research question. It’s just instead of it being an animal – I mean people are animals but it’s people and you can ask them questions and they tell you things, which dolphins and whatever else don’t do as clearly. Sometimes people don’t either to be honest.

[00:05:51]
Jamin Brazil: I really want to dive in on this topic but unfortunately we don’t have two hours. They’ve limited our time. It would be fun to meet in person at some point and chat about that journey because I do – it’s interesting how the digital is now – thinking I have a five-year old. He’s in school and in California the school system that we’re in is all fully remote. So he’s spending literally five hours a day on Zoom calls with his teacher and some of it is one-on-one; most of it is in group. And it’s been really interesting from an ethnography perspective just me as a dad sitting back watching his learning, the things that are going really well and actually probably accelerating relative to a classroom environment versus otherwise maybe not going as well or things that he may be missing. But regardless of the outcome on the learning side, his comfortability with technology is going to be exponentially higher right now let alone in two, three, four, etc. years. There’s a redefinition that’s happening with us in the way that we feel about how technology connects us. So like in that framework – this is a little bit of an older stat but it was projected that 2020 the global Esports market value was around a billion dollars and is project to be $1.6 billion in 2023, which is huge growth, massive growth in a two-year period. Let’s kind of level set for our audience right now. What is Esports?

[00:07:20]
Laura Levy: Yeah, so there’s – I think that question has two answers. So the simple answer is Esports are just games played at a competitive level, often to huge audiences pre-COVID, live arenas, Madison Square Garden sold out, seats of a venue of 22,000. They didn’t use the entire arena but for the Overwatch finals in that first year they sold those tickets out like months ahead. So that’s the easy answer. But what I like to communicate about what Esports is, especially to students because I teach some classes around Esports at Georgia Tech. In their head, and this is what Esports is now, modern Esports is part rock concert and part Super Bowl. There’s pageantry, there’s fireworks, DJ Collins there, there’s super-fancy graphics, they pipe up the players as superstars, but really the first time we had an esport was like in the 1950s, a game called Spacewar. That was played on basically a souped-up oscilloscope. And there’s some really cool photos that you can find, black and white photos of clearly engineers, pencil-thin ties, white button-up shirts and socks, and they’re all crowded around this computer that takes up an entire wall playing this game. And then from there we get things that look more familiar to today. So you can find pictures from the 1980s, world championships people are doing some of the first land parties. So even though Esports to us especially in the western hemisphere is kind of new and feels really souped-up it’s been around for a while, particularly like in the Asian market Esports has been huge for 20 years. So it seems new to us now but it’s been around for a minute.

[00:09:11]
Jamin Brazil: Interesting. Great. Well let’s get our next guest in. Emma Varjo is the UX lead for Frozenbyte. Frozenbyte was founded in 2001 and is headquartered in Helsinki, Finland. Now with now over 110 employees, Frozenbyte has 11 published titles, most recently Boreal Blade, which is a teen-based melee fighting game, which is focused on player versus player. They also have the Trine series, which is a best-selling game in the adventure genre. And they are scheduled to launch Starbase, which is a space-based MMO with a fully destructible and infinitely expanding universe focused on building and designing spaceships and stations, exploration, resource gathering, crafting, trading, and combat, which all those things sound fantastic to me in context of like my MMO background, which starts way before there was even a Gooey back in the ROM Mud days if that gives you any context. Prior to joining Frozenbyte, Emma has served as both a software developer and software designer. Emma, thanks for being on the Happy Market Research podcast.

[00:10:29]
Emma Varjo: Thank you so much for having me. It’s super-exciting to be here.

[00:10:31]
Jamin Brazil: As always we have this standard question just to give a little bit of context of who you are. Tell us a little bit about your parents and how they impacted what you do today.

[00:10:40]
Emma Varjo: Gosh, actually my dad has done a long career in marketing and is now teaching that at a vocational school I think. And my mom has been doing a lot of clerical things and when I was younger she was a stay-at-home mom for much of my young life. And yeah, I guess if I had to guess where my curiosity for how people and things work and to make things better for them is possibly that’s one explanation for it.

[00:11:11]
Jamin Brazil: So marketing is interesting from your father specifically relevant to like relevancy inside of communities because that is really the underpinnings of marketing. Do you guys talk much about your current work and how marketing tactics, those kinds of things could be applied or are being applied to growing your community?

[00:11:35]
Emma Varjo: We do talk about work but not on that level that much partially because my dad was a marketing director towards the end of his industry career and he was more about number crunching and managing people instead of doing the actual marketing. And I support marketing here and there but my focus is slightly elsewhere. So while we’re both interested in similar things, it’s not exactly the same. So yeah.

[00:12:04]
Jamin Brazil: Yeah, that makes perfect sense. Let’s kind of jump into the discussion now that I’ve got both of you here. When do Esports companies reach out, Laura, to you? In other words, what is the business problem that they’re trying to solve?

[00:12:21]
Laura Levy: Yeah. So I think there are two main challenges the industry is looking at right now for Esports franchises and the reason why we work with our industry collaborations. The first one is fan acquisition. So especially now with COVID, we’re kind of starved for watching anything at all. So we’ve seen traditional sports try things like have the NASCAR drivers play this in a NASCAR-esq type of game. So with things like Overwatch, we work with Atlanta Reign, which is the Overwatch franchise for Atlanta. They’re working on this tribalism piece. So if I live in Atlanta, it’s painful, particularly for us but I cheer for the Hawks or I cheer for the Falcons or Atlanta United. There’s a lot of people that see Atlanta Reign and go hey, I want to get into that but I’ve never played Overwatch. And I don’t know what I’m looking at. And a quote when you’re doing studies of novice viewers, people who have never seen Overwatch league streams before, the quote was, “Everything is happening all the time.” And they want to get into it because they want to have that thing but they don’t know what’s going on. So in terms of fan capture and fan acquisition how can we create maybe tools or experiences or activations so that people can feel connected to a team, a franchise, a game because ultimately the goal of Esports is really to drive people to play that game more. That’s kind of the business connection. And the other piece is player health and safety. We know a lot about sports science and how to support a team like a college football team, a professional NFL football team, but we don’t know what that looks like for Esports athletes and they’re younger. Many of them, maybe they didn’t do so well in high school, they didn’t go to college, they started when they were 14. They were pulled out of their parents’ houses. They don’t know how to eat right, the benefits of exercise, managing strong emotions. So how do we construct teams that support these players so they don’t burn out and so that we can get the best performance. Because obviously a franchise wants to have a very competitive team. That drives viewers and championship titles but there needs to be some research there to understand like how do we even do this. So those are the two main things that the Esports industry has contacted us about at least.

[00:14:51]
Jamin Brazil: Emma, anything to add, any other reasons why or maybe even furthering these points that Laura just mentioned?

[00:14:58]
Emma Varjo: Not really. I work in a very different space. So that was – it’s always interesting to talk to Laura and hear that side of things because that’s not what we think about at all really.

[00:15:10]
Jamin Brazil: When is research employed inside of Frozenbyte?

[00:15:13]
Emma Varjo: We have a fairly solid idea of what kind of games we’re going to make. So it’s more heavily geared toward – we do them more heavily toward the end of production cycle. And it’s always design and the game first. So we do very little pure marketing research for example. We do some of that but it’s more about how does this game play, how do people see things, how do they react to things, and how do we set their expectations, that sort of thing. It’s very much focused on the game and then we go from there.

[00:15:49]
Jamin Brazil: So a lot of kind of level testing where.

[00:15:53]
Emma Varjo: Yeah.

[00:15:54]
Jamin Brazil: Where you’re – in a simplistic way I’m going to try to explain it and you’ll have to correct me please.

[00:15:59]
Emma Varjo: Yeah, I went on very high level. You can go more detailed.

[00:16:02]
Jamin Brazil: No, that was good. So it’s kind of like level one, players are getting through level one so quickly and then – or they’re not. They might be getting out and then level two and so on and so forth so that you’re kind of maximizing that transition from thing to thing.

[00:16:17]
Emma Varjo: Exactly.

[00:16:17]
Jamin Brazil: Maintaining the engagement through the next.

[00:16:21]
Emma Varjo: Yeah. And seeing – because usually the first few levels are tutorials of how you’re introducing the mechanics and everything to the players. Do they understand everything, do they even see everything because there have been instances where we teach a mechanic and they’ve just missed it completely. And then later on they’re confused and seven levels in when they need that to proceed and they just don’t have that skill on their mind. And also the pacing of how things are going forward once you get past the tutorials and everything.

[00:16:49]
Jamin Brazil: How do you guys gather that feedback? Is it directly inside of the system or is it like post the experience then they report back to you where they had trouble or?

[00:16:59]
Emma Varjo: Our most utilized method is to invite people into our offices. Obviously we cannot do that now and we’re kind of scrambling. But we invite them in and have them play the game and we’re recording through – the look on their face but also the screen, what they’re doing and we have people monitoring what’s going on. So we see immediately if there’s a problem and we can take note of that, oh we need to fix this but also if there’s a game-breaking bug and they get stuck. So people can come in and help them. But mostly it’s hands-off, just watching them play, taking notes, and then like taking them and moving on and fixing things.

[00:17:38]
Jamin Brazil: I’m really excited about the MMO. Do you guys have a – because this is like the only kind of game I’ll ever play now. And the reason why, which I think is interesting is it creates the human connection with – in a video game environment, which I find very attractive and fun. Yeah, and apparently I’m not the only one. So as you’re getting ready to launch, I assume you’ve done some betas.

[00:18:01]
Emma Varjo: Yeah, we’re currently in post-alpha. So prior to this we did some testing on like are we getting the tech right and all these sorts of things. And more contained things and that’s when we invited people over. But we’ve started post-alpha toward the end of spring. I don’t remember the exact date and are progressing toward early access launch. But now it’s a contained number of people, still a lot more than we had in just our studio, which we did like huge play sessions on Friday afternoons like everyone’s playing just to see if we crash this. But now it’s hundreds more players and we have actually a tool built within the game so they can take a screen shot and say this is an issue that I’m having or this is where I’m having an issue. So we get that feedback as easily as possible from our players.

[00:18:53]
Jamin Brazil: I think the integration of consumer feedback inside of the systems that we’re using – the video game industry has largely pioneered that and now that we’re moving more and more into a digital interaction, a good example is this event. The three of us would have met – historically we would have met in person and done it on a stage at IIEX and now that isn’t feasible. But at the same time we’re able to – like if there is an issue with a product that we’re using to stream, and you’re going to see more and more introductions of what is your experience like and ability to be able to get that consumer feedback. I did have one last question on the MMO thing just out of personal curiosity so I do apologize about the diversions. Space MMOs have been tried in the past, right? And at varying levels of success. I’m really interested, were there lessons learned from some of the other products that or titles that had gone out that you guys have employed inside of Starbase?

[00:19:53]
Emma Varjo: Yes and no. These sorts of games, they are in development for a very, very long time. And interest for them, and they started popping up around the same time. So they were all in development for a good while. Obviously we’re coming out a bit later than some, so we’ve heard and been able to see how they build off of it first. But a lot of the core things that we’ve had and the mechanics that we’ve built and the tech has been there before the others have been launched. But obviously as they come out, we see how they do things like tutorials and what things look like maybe, so we can riff off of that a little bit, but not the core things.

[00:20:35]
Jamin Brazil: Well, hopefully the audience will take time to Google Starbase and sign up. Maybe some of them will get into a beta.

[00:20:43]
Emma Varjo: Hopefully, yes. I’m really glad that you’re interested in it. I’m super excited. I’m just grinning like a goof when you’re talking about it.

[00:20:53]
Jamin Brazil: Anyway, we can talk a whole episode on this topic. But we won’t. We will stay on topic or get back on topic, I should say. So Emma, who in the organization is using insights?

[00:21:05]
Emma Varjo: I think it’s mostly through designers, or that’s the mindset that we do – we have. So everything is like, “How do we design the game to be better?” But obviously once we have the insights, they’re public for the entire company, and then they’re used by marketing and management and everybody else. But the idea is what kind of experience are the players having currently? How are we going to make it better? And then everyone can take their takes – I forgot the word, but their ideas from there, like how are they going to go forward?

[00:21:40]
Jamin Brazil: When you’re developing a new game, do you have injection points of consumer insights? Is it part of the milestone or part of the build-measure-learn, however it is that you guys are framing out the innovation wheel?

[00:21:52]
Emma Varjo: That depends a little bit. We sometimes have agreements with a publisher, and then they have a set schedule for milestones. And then they might impose things for us. Usually they want to do their own insights things with a deliverable that we give them. But with us, it’s more a touch-and-go thing like, “Now it feels like we could do something with this, so let’s start testing.” And then it’s not as scientific as it could be, but it works for us.

[00:22:23]
Jamin Brazil: Laura, how about yourself? Is there – the companies that you’ve worked with, are there points like static points inside of their processes where they’re injecting consumer insights?

[00:22:33]
Laura Levy: Yes. So for us, mostly we follow – it’s called a user-centered design process. So often when we have that first connection with industry or we’re starting a project, we’re trying to do that with them. People are not super great at telling you what their actual challenges or barriers are. And sometimes both industry or users will get latched on to some cool thing they saw in a YouTube video and they’re like, “What if we built this AR experience?” And we are huge AR proponents, but we try to go in very platform-agnostic and say, “What are you actually trying to do?” And then we pick the technology that fits that. We build a little bit. We show that target demographic. We get some feedback, and we do that multiple times. Because we’re more research-focused for us, even though I work primarily with industry, we have to have a research component to the work that we do. So we’re able to do that back-and-forth design process, and I don’t know. It depends on the project. But you might do that four or five times depending on what you’re building or what you’re testing. We worked with the Atlanta Braves to build an AR experience around the new stadium because we moved from Ted Turner Stadium, which has a lot of aura, a lot of history behind it. You can see the skyline from the stadium. But then they moved out north, not even technically Atlanta, to a brand-new stadium. It has no history. It has no vibe to it. So how do you get fans engaged with this stadium that – it’s clean. It doesn’t have anything – there’s no memories there. How can you build that? So we did a bunch of focus groups and interviews, showed them stuff, tried to figure out “how do people relate to a franchise or to a team in general?” A lot of this ports directly to Esports too. When someone doesn’t know a team or a game, what are those touchpoints they make? And often it’s with individual players. So those kinds of research insights, I guess you would call them, will then drive that product. And we’ll just do it until we have a thing that is done, and then we start the next project, which might be building onto it or building around it.

[00:24:51]
Jamin Brazil: So it’s like rapid prototyping, right?

[00:24:54]
Laura Levy: Exactly, yes. It is rapid prototyping. Some of the prototypes are rough. They’re literally paper. It looks like a game, but you can move all of the elements around on paper. And we’ll redesign menus or skinning for it. And it’s really nice because you don’t sink a lot of time into building something. You just have some art assets. You put it down on a table. Little bit harder now with COVID, but we’ve been exploring doing that in Google Slides, tools that are not meant for this, so that participants can have that hands-on piece, just safe.

[00:25:28]
Jamin Brazil: I’m seeing a lot of that, like the taking technology over for this use case that you’re describing of collaboration without the – specifically, Google Slides is a great example because it obviously wasn’t intended for this purpose. But it’s perfect for it. Zoom is another really good example of a platform for conducting IDIs. You’ve got closed caption if you need it. You’ve got real-time transcripts. You can create a – I sound like an advertisement for Zoom. I don’t mean it like that. But my point is that there’s a lot of cycles that are being applied right now to figure out what tools researchers can use in order to get the research done. We’re a scrappy industry though, so it doesn’t surprise me to be quite honest. Emma, thinking about research, you spend money on UX. It’s like a cost center for the business. There’s always an expectation of a return, at least from the executive team.

[00:26:32]
Emma Varjo: Yes.

[00:26:32]
Jamin Brazil: CFO, cough-cough. As an internal researcher, how do you measure the ROI?

[00:26:39]
Emma Varjo: Actually, this is not – this is going to be a very unsatisfying answer. But we really don’t. Our company started very small, and while it has grown a lot, it still has retained some of the heart of it. And a big part of it is that we want to focus on games and not on the money, which is – that’s one of the clearest things that we see, is the money is hidden in everyday things that we do. Everyone has access to finances and everything if they want to, but we’re not given “this project has a strict budget and you have to stick with this” or anything. It’s always more about needs and what the game needs and what the resources are at that moment. And we have then higher-ups that we check with and balance this in a wonderful way without really mentioning money numbers at any point. It’s more about hours and people that we talk about. And it’s kind of weird coming from other industries where that was a big thing. But it’s also very relaxed and liberating in a sense.

[00:27:48]
Jamin Brazil: Congratulations. That sounds like a really nice environment to be in. It sounds like consumer insights are more of like a core foundational element for – it’s more of like a cultural characteristic or tenet, or I’m not sure what the right word is, but core value for the company. Is that –

[00:28:07]
Emma Varjo: It is, and it’s on the same page as everything else like programming, which is always – everyone has too few programmers on their games, right? But it’s on the same level. There are programmers, and they don’t have to think about money in the same way that we don’t have to think about money. And we’re all doing important things for the game, so it’s a lovely balance on that.

[00:28:26]
Jamin Brazil: Give me some context of where you sit organizationally. Are you working directly with the developers or marketing, or who are your stakeholders?

[00:28:36]
Emma Varjo: It depends on the project. Usually, the easiest way to think about it is project management. They’re my main stakeholders. But where I actually do my work is more with the designers of each team. And then we do collaboration with marketing when our intentions and needs align. But it’s more like we just bubble about or bumble about doing our own thing, and then trust management to give us direction of “you’re needed here, so ship this way.” And it’s more about the everyday interactions with the teams to have a bead on it myself as well. So I can sometimes go to project management and say, “Maybe we should be testing this thing this way now, because I know that the devs are talking about this.” And they’re like, “Wow, that’s cool.” Or sometimes they come to me and say, “This team really needs help with this thing, and we need to research that.” So it’s an interesting way of doing things.

[00:29:30]
Jamin Brazil: Yes, for sure.

[00:29:31]
Emma Varjo: I can tell you that.

[00:29:33]
Jamin Brazil: Well, if it’s not interesting then we’re not having fun. You have a lot of titles that you’ve launched. Do titles in the video game world – do they get sunset, and then kind of moved off? Or is there some kind of ongoing legacy that is applied from a research perspective to those games?

[00:29:54]
Emma Varjo: It’s different from game development way, because we’re – Trine is a great example. It has four installations, so when we start – let’s say we were making a new Trine game. We would have to look at what the history is there like what the mechanics are, how people received it. Also technical things. There would be code that we would try to import as much, so we would incur a technical debt from there, but also create new things. Research-wise, we can start on a cleaner slate. Of course we look at previous research done, but it’s also very free of “what’s happening with this game?” So we don’t necessarily have to focus on what’s happened before, although it obviously helps both on the design side and research side what’s been done before so we can reflect on those. But it’s not as closely tied to past as other disciplines are.

[00:30:50]
Jamin Brazil: Laura, how about yourself? Do your customers – do they think about the ROI when they’re engaging with you, whether it’s rapid prototyping or something else?

[00:30:59]
Laura Levy: Yes. I think sometimes there are – mostly what my group does – there are two kinds of projects. There’s projects like especially in the health field, and we build also serious – they’re called “serious games.” Nobody really likes that phrase because –

[00:31:15]
Jamin Brazil: No. I already don’t want to play it.

[00:31:18]
Laura Levy: Yes, exactly. And educational games and stroke rehabilitation games and all of that, they still should be fun because we know that the traditional method is almost not fun most of the time. But if we’re doing an applied thing like that, there’s some quick wins that we can make to say, “This is definitely going to help people in poverty connect with the resources that they need,” that sort of thing. But the other types of projects that we do are much more experimental, like a lot of – we work with Google. So they’ll come to us and they’ll say something like, “We think in the future, people will be engaging with technology in this way. But we don’t know what that tech looks like.” Or “we have that.” So Google Glass, the technical components were developed at Georgia Tech with Thad Starner and researchers from my group. They’re like, “We built this thing, and we don’t know how people are going to use it. So can you come up with some studies around it and see what use cases are, how that integrates with people’s lives?” So those studies tend to be a little bit more like a company saying, “We’re going to give you this money, and we know that you’re going to do some interesting research with it.” But it might be a bust. There’s always going to be some kind of research or insight or finding that comes out of it that’s really important. But because we often work on projects that are cutting-edge or bleeding-edge or any of those buzzwords, sometimes you just don’t really know what it’s going to produce. So it’s hard to really draw a straight line between “we gave these researchers money, and this is our ROI based on what they did.” It’s more like we don’t know what people are doing with wearables and gaming or – I don’t know. Fitness games are a really challenging space. We’re all kind of lazy. We just – everybody stands up and plays with the Wii, and literally or closer to literally bowls. And then we sit on the couch and we just flick our wrists, and we’re playing tennis. So sometimes those connections are not as clear for, “What does this actually do in terms of changing the way games are built or the way this technology works?” Sometimes it’s a little bit more fuzzy.

[00:33:30]
Jamin Brazil: What’s interesting there especially in context on rapid prototyping is the big learning might just be “don’t place the bet.” Or if you’re like Pokémon Go, when Nintendo launched that, that game is still an active userbase and –

[00:33:43]
Laura Levy: I play every day.

[00:33:44]
Jamin Brazil: So there’s a – it’s a really interesting – when you think about the actual ROI on research, this is kind of like my challenge to my colleagues in market research specifically and user experience and CX, is it helps us as agencies to think about what the actual – whether it’s measured in savings or upside, is for the research. What is that actual outcome going to be? And that can help a lot of times the clients recommunicate that back to their internal stakeholders as well, because a lot of times companies aren’t actually framing it out that way. We’re seeing a ton in the VR space. I was really surprised. At my last company, we had one of the developers that got a Google Glass. This is, what, almost eight years ago now? And so it was super fun having that thing right there. And the applications for Google Glass seem infinite, all the things like pre-flight checklists for pilots to surgical checklists in general, I think. But then also, it’s such a more elegant way of getting information versus something like this where you’re looking at a watch, which is very distracting. And one of the things that I thought would be really cool is if you and I could meet in person, not know each other, but have each other’s LinkedIn profile or a highlight of that pulled up so we have some immediate context, creating this shortcut. Then all of a sudden, now I think it’s going to get adopted by Tinder. But –

[00:35:25]
Laura Levy: But this is a good example that you bring up in terms of research. Glass is a great example. Glass is alive and well in enterprise, car factories, factories in general. But what the research could’ve really supported is this understanding of what we call socio-technological aspects. People really don’t like that camera that is just pointing at them. And we saw that have a huge impact on society. Movie theaters ban them. There was a term that described people wearing Glass, because the technical component was super sound. It is a cool piece of technology that people can tinker and play with and customize. But it really creeps people out and it makes them feel uncomfortable. Even the wearer kind of feels like, “Man, I feel really weird. I’m one out of 200 people that’s wearing this thing.” So if there’d be some maybe more research around that, there are ways that you can design that little camera eye to give some clear indications like “hey, it’s not on,” or whatever. Some other kinds of affordances so people understand and don’t feel like they’re being surveilled. And that was one of the main downfalls of Glass. But as a unit, it’s actually a very cool piece of tech.

[00:36:45]
Jamin Brazil: Yes. You see that with Snap being classified as a camera company as opposed to a social media platform with their S-1. So I think at first – and of course they have their own Glass. It doesn’t have the flip-out thing. It’s just like glasses with a camera embedded in it. They’re not for me, but I’m almost 50. But there is this overarching theme of comfortability with technology that I think is really driving a lot of the things that were maybe far-reaching and bringing that in. And so when I think about time on game, so how much time I spend or a person, a player spends – you could go with like Ready Player One as the extreme example of an immersive environment. But if I could – the shortcut is maybe it is – I don’t know how it works. But it just feels to me like the next generation could be in class while sitting outside at the beach or in a park or whatever. And the way that that could be accomplished is probably a lot less of having this laptop in front of them, and a lot more of having this kind of lens with some writing utensil or what have you. So we’re in this interesting redefinition of what normal is at a society level. Do you think that the social norms will be readjusted so that something like Glass becomes integrated into society anytime soon?

[00:38:17]
Laura Levy: I hope so. We have seen shifts in wearables, and I have colleagues that only do wearable work. We have grants to do wearables for assistive tech, which I think is a really promising space. So for people with perceptual disabilities or cognitive disabilities, wearables that can support the way that they interact and move through our world is super promising. But we still have to get over this hurdle of that socio-technical piece, particularly for assistive tech. If I’m blind and I have a wearable sleeve that I use to interact with a kiosk or my own devices, maybe I don’t want people to see that sleeve. Or maybe I do want them to so that they know that maybe I need assistance, or – because we find that people who have visual impairment sometimes want to communicate to the world like, “Hey, I need help right now.” And they’ll pull out their cane when they don’t need it, because they’re hoping maybe someone tells them what bus just pulled up. And other times of course they want to keep it hidden because people will come up to them and grab them by the arm and be like, “Let me help you across the street.” We have to really consider the form of these, whether it’s on our face or if people make pendants that you wear around your neck so it’s not exactly in your eyesight, but it’s someplace else. Jewelry wearables too that do similar things to Glass. We have to consider what this looks like so that it’s preferable for both the wearer and the person with that wearer, because otherwise we’re just going to keep making tech that creeps people out. Or it’s onerous to wear too. Glass is kind of heavy if you wear it for a while. It gets hot. You’ve worn one that’s actively working. That little tiny power unit here gets kind of hot. So we have to think a lot about just the acceptability of what it looks like and what people’s mental model is of what it’s doing too. Is that camera on. Is it not on? How do I tell? What is this person looking at? Are they looking at my LinkedIn? I haven’t updated it in years. That sort of thing. Not me personally. But a lot of-

[00:40:32]
Jamin Brazil: Are they single. The- Or which one do they care about.

[00:40:38]
Emma Varjo: Back to that creepy thing?

[00:40:39]
Jamin Brazil: No, exactly.

[00:40:40]
Laura Levy: [CROSSTALK] population. Women are probably like no, I don’t want someone to be able to know who I am or where I’m from. Being able to have a high level of control over what you are sharing with the world or not is super important.

[00:40:55]
Jamin Brazil: And all the PI around that. Emma the Esports, myself for example, I’m a bit college sports. Huge fan. Everything’s been shut down. And even though I have an avid gaming life and my family does too we’re really not, we’re connected to different streamers. But we’re not connected at an Esport level. In other words there’s not a, in February there was the Super Bowl normally or whatever kind of a thing. Do you think we move towards that, more of this, these iconic events that take place?

[00:41:37]
Emma Varjo: I can only talk about through my personal experience I haven’t really been watching the space as a professional but my friend group, we always gather around for the international for Dota 2. So that’s definitely a thing that we do. There’s the same thing for League of Legends. A similar tournament every year. And also the smaller ones. But the big international things and the main one is, they’re big things for some games at least. But there’s a problem of what team do you route for. Because I could say, and I’ve seen people actually out jogging that wear say Team Liquid shirt. But I don’t know which league they’re watching. Are they following CS or Counter Strike or are they watching Defense of The Ancients two? Which game are they playing because now there are these big corporations that sponsor many different teams for many different games. We don’t have say Seahawks, which is my favorite NFL team. Once you say that you immediately know which team, which sport but we don’t have the same thing for Esports yet.

[00:42:46]
Jamin Brazil: It feels a lot more, the little tribes as opposed to more of these bigger things. Laura do you have, do you think that Esports is, people are going to start choosing Esports over regular sports? Do you think that it feels a little bit more like a zero-sum game in context of time?

[00:43:05]
Laura Levy: No. I think we’re seeing a blurring of lines between traditional sports and Esports. And this is exciting I think. So a colleague of mine at the University of Utah, Roger Altizer has said that Esports is us getting to do traditional sports over again. So we can look at the things that went well in traditional sports from how you support fan communities to the technologies that support the way that we experience those, the way the live experience is constructed we can port that into Esports and vice versa. So I think really they build off of each other. And in the same way that we’ve seen the breakdown of what it means to be a nerd in the past 30 years I think we’ll see that for fans of Esports. Because if you read comics in the 1980s or prior you were a little untouchable in a lot of different kinds of social circles. But now the biggest movies are comic book movies and there’s some friction there for people who were like, hey I kind of suffered for loving this thing as a kid, being bullied. And now any yahoo can wear an Avengers shirt and not get teased for it. I think we’re starting to see that breakdown too for an Esports athlete is not just some neck beard in their mom’s basement playing whatever StarCraft. We’re seeing this blend of hey that is a feat of athleticism. These sports athletes they’re APMs. Their actions per minute are crazy. It requires a high level of cognitive ability and reaction time and special awareness and game sense. So I think you’ll have traditional sport fans being like hey, football season is over but the Overwatch league is happening and I live in Atlanta. That whole piece that they’re trying to do for the localism and vice versa. The Overwatch league season is about to end in a couple of weeks but there’s other sports, things that are going to happen. And maybe someone’s like hey I kind of miss cheering a team. I’m going to see what The Hawks are about or we may- Sorry, Thrashers. I miss having a hockey team here. But they might glom on to a traditional sport just to have something to have that kind of social connection, the relation with the city that you’re in, the fun of having a viewer experience of some kind of competition. So I hope that it joins us.

[00:45:34]
Jamin Brazil: You kind of see it with the NFL and of course other sports. But there’s a, the Madden franchise for- And so they’ll always do the Super Bowl and Madden first to see who won. There’s also this really interesting kind of player worth, real life player worth that has materialized in the NFL. So whatever player gets on the title of Madden, even though now it’s all digital based, is that sort of the poster child for Madden. All of a sudden their relative value as an athlete now materially increases inside of that, absolutely increases inside of their NFL career. So it’s funny for me to see this connection of a digital pretend to players getting really upset that their, they aren’t as- They aren’t scored as fast on Madden as their counterpart.

[00:46:33]
Laura Levy: And also we’re seeing the lines blur where the Atlanta Hawks, they bought a 2K team. So they have a team of Esports athletes playing NBA2K as The Hawks. So that’s an even more kind of literal overlay of a traditional sports team owning an Esports team mimicking themselves and we’re seeing that all over the country. I don’t know if FIFA maybe in Europe is similar are there sport clubs buying FIFA types of teams. I don’t know about that.

[00:47:06]
Emma Varjo: I’ve heard a story, and I’ve completely forgotten the name of the game, but there was this English game that was centered on football and they were doing what Madden’s doing now for NFL. They were mimicking the leagues and they hired actual fans to go to the games and see how people were performing and they had stats and this was back in the day before the teams themselves had these stats. So they actually went back to the game developers to ask for these stats and then they started, this is how they started figuring out how to buy people from other teams to their thing. And they had their own systems before but it was more touchy feely and now they have stats from a game that they use for this. So it has been going back and forth for a long time.

[00:47:49]
Jamin Brazil: That’s so interesting. EA has a whole division around basically decomposing the athlete in terms of how they, their stats. I don’t know what the stats are. Agility, strength, size, whatever. And then and those guys or gals, people, are really important and sometimes loved and sometimes hated based on how you get ranked. It’s kind of interesting. So just to clarify you both think that it is less of a zero-sum game in terms of Esports stealing share from traditional sports and more of a collaborative relative growth in both categories?

[00:48:26]
Laura Levy: Yes. I definitely think so and I am excited about what Esports borrows from traditional sports especially in terms of technology. Because there’s things in traditional sports, that experience that we take for granted. In American football you have the first down yard line. That’s augmented reality. And that’s been around for a while. We see a lot of computer-generated stuff in a broadcast sports scene. You can kind of argue that watching it at home is a better supported experience than watching live in that stadium. It’s not the same feeling but you get a lot more information because you have this Hud. So with Esports we have this perfect knowledge of a game whereas in football they will replay the tape. Well was his foot in or out or what was that? Look at it from multiple angles. With Esports there’s no question what happened because we have the build and we can run it back. But it also means we can do things like embed in the same way, still augmented reality. Embed ads into the game and not to create this dystopic marketing nightmare. It still might be preferable for a billboard in a Twitch stream advertising specifically to me a specific graphics card, but to you a mouse. That might be preferable to be in the game, doesn’t impact the athletes to this ad sponsor saying why is this match going on for 30 minutes? I have not seen the ad that I paid for. Having to wait for those breaks and then that might encourage businesses to invest more in Esports and that bumps it up but it’s same tech that we’re porting from traditional sports into this game space. So I think it’s going to kind of ratchet both. I’m excited about that.

[00:50:20]
Jamin Brazil: Well we are at the end of our time. So I’d like to go out on one question to both of you. Laura let’s start with you. What is your personal motto?

[00:50:29]
Laura Levy: So my personal motto is borrowed from Russell Kirsch who created the pixel, the graphic length unit. And he would say that, “Nothing is boring if you ask enough questions. And you have to keep asking questions and then stuff becomes really interesting.” I think that’s very important in research because sometimes you get locked in on you think that your target demographic, your user, your consumer already knows this sort of thing. Or you think you know what they’re thinking about. You have a locked in mental model, but the purpose of research is to do this. You have to just keep asking questions even if you think you know you still have to do that sit down. Do an interview. Do that focus group. So that’s what I try to think about all the time. I’ll think I know something. I’m well let’s actually just create an interview and see if that’s right.

[00:51:22]
Jamin Brazil: Emma, what about yourself? What is your personal motto?

[00:51:25]
Emma Varjo: Mine’s super similar and now I’m thinking I’m stealing it from someone that I just can’t credit them. It’s, “Stay curious.” Because there’s always more to learn whether it’s a thing, a topic, or a method, a person, or an event, or whatever. Be it work of how are my players reflecting on this and feeling about this thing. Or you come home and what’s my partner thinking about today. Everything is, you should always be asking questions and being curious of what’s going on around you because once you get complacent there are going to be problems.

[00:52:00]
Jamin Brazil: My guest today have been Laura Levy and Emma Varjo. Thank you both for being on the Happy Market Research Podcast today.

[00:52:09]
Emma Varjo: Thanks for having us.

[00:52:10]
Laura Levy: Thank you. Thank you very much.

[00:52:12]
Jamin Brazil: Everybody else, it has been a pleasure having you join us for this very first video, what it looks like when we do a podcast with adding video, which has been interesting for me and a little bit unnerving. But it’s been great seeing our guests faces today. Ladies again, really thank you very much. As always I appreciate you tuning in. If you have any questions or want to reach out to me please do. You can find me on LinkedIn, Jamin Brazil or on probably any other social platform except Snapchat and maybe TikTok. Not as active there either. Have a good rest of your day.