jaminbrazil

Part 10 of 10: Tips to Writing The Perfect Survey

Lessons Learned from Programming 2,000 Surveys Over 20 Years

You can find the start of this series here. As you are going through this article, please 👍 and share if you find things useful. 

Tip 10: Survey Length

Keep your survey to 4 minutes. 

Why 4 minutes? 

How much of your life changes in 4 minutes? 

In the last 4 minutes of writing this post my baby climbed over me, my 11 year old showed me her iPhone game score, Toy Story crescendo-ed and I responded to a text from my best friend.

No alt text provided for this image

Literally the moment I referenced 🙂

This will force you to be crafty with your questions. Here is an example of what I mean…

Q10. Do you drink beer? 

  • Yes 
  • No

–page break–

[if yes, ask Q11]

Q11. Which beer do you use most often? 

  • Bud 
  • Coors 
  • Miller
  • Other, please specify

We can combine both Q10 and Q11…

Q11. Which beer do you use most often? 

  • I don’t drink beer <– was Q10
  • Bud 
  • Coors 
  • Miller
  • Other, please specify

Pro Tip: It takes a respondent about 1 minute to answer 3 questions. 3 questions per minute * 4 minutes = 12 questions

CONCLUSION

If you are interested in my next project, you can subscribe at begreat.io. As always, I hope you found this content useful and wish you only the best. If you’d like to add to the conversation…please drop a note below. It’d be fun to chat. 👇👇👇

It would mean a ton if you would share this article on TwitterFacebook, and LinkedIn!

#marketresearch #userexperience

Part 9 of 10: Tips to Writing The Perfect Survey

Lessons Learned from Programming 2,000 Surveys Over 20 Years

You can find the start of this series here. As you are going through this article, please 👍 and share if you find things useful. 

Tip 9: How to Word Your Questions

Questions are the absolutely hardest part of writing a survey and there are volumes on how to word questions so that you get the intended answer. I have been in more than a few board presentations where a board member wants to know the actual question that was asked of respondents.

“Judge a man by his questions rather than by his answers.” — Voltaire

Consider this question…

Would you be willing to pay a lot more for a Nike shoe endorsed by Michael Jordan vs the same Nike shoe w/o the endorsement? 

That was basically the question asked by researchers when Nike was deciding if Jordan’s endorsement was worth it. The answer they got back was, “No way would I pay more for a shoe just because it has someone’s name on it.”

In the second round of research they asked the same question with a twist…

Which of these two shoes would you purchase? The Nike Air Jordan’s for $68 or Nike for $30?

And that, my friends, was the right question. Instead of being directly asked about the value they were asked about the product. It might seem like a small difference but it has HUGE implications to your data.

“I’m sorry, my responses are limited. You must ask the right question.” — Dr. Lanning’s Hologram

For now, just keep in mind that words often times mean different things to each of us and it is important to make sure your question is crystal clear. I’ll frequently have my 11 year old read a survey to ensure it is addressing all angles.

CONCLUSION

If you are interested in my next project, you can subscribe at begreat.io. As always, I hope you found this content useful and wish you only the best. If you’d like to add to the conversation…please drop a note below. It’d be fun to chat. 👇👇👇

It would mean a ton if you would share this article on TwitterFacebook, and LinkedIn!

#marketresearch #userexperience

Part 8 of 10: Tips to Writing The Perfect Survey

Lessons Learned from Programming 2,000 Surveys Over 20 Years

You can find the start of this series here. As you are going through this article, please 👍 and share if you find things useful. 

Tip 8: Profile

The easiest to understand but the hardest to get right. These questions almost always include gender, age, ethnicity, income and education to name a few.

These add length to your survey and can be a distraction. Ask only the questions that are useful. How do you know which ones are useful and which are not? Easy. Can and will you target the customer based on the profile?

In my bar survey, I am not going to target females and males differently so we’ll leave out the gender question. In short, kill all “nice to have” questions.

Additional profile questions include:

  1. Behavioral: How frequently do you do X or visit Y. Basically, this set of questions are uncovering your product utilization to help understand channel usage.
  2. Firmographic: What demographics are to people, firmographics are to organizations. However, Webster (2005), suggested that the term “firmographics” is a combination of demographics and geographics.
  3. Phycographic: Interests, hobbies, opinions, etc.

CONCLUSION

If you are interested in my next project, you can subscribe at begreat.io. As always, I hope you found this content useful and wish you only the best. If you’d like to add to the conversation…please drop a note below. It’d be fun to chat. 👇👇👇

It would mean a ton if you would share this article on TwitterFacebook, and LinkedIn!

#marketresearch #userexperience

Part 7 of 10: Tips to Writing The Perfect Survey

Lessons Learned from Programming 2,000 Surveys Over 20 Years

You can find the start of this series here. As you are going through this article, please 👍 and share if you find things useful. 

Tip 7: Diagnostics

These questions are the core of your survey as they directly answer your objectives.

For our example survey, I have already identified that my diagnostic questions will be based off the Van Westendorp’s Price Sensitivity Meter. So, my diagnostic questions will follow those guidelines.

The good news is that it is highly likely there is a published set of diagnostic questions you can tailor to fit your specific objective. SurveyMonkey is a great resource for free surveys that already have the diagnostic questions built for you.

CONCLUSION

If you are interested in my next project, you can subscribe at begreat.io. As always, I hope you found this content useful and wish you only the best. If you’d like to add to the conversation…please drop a note below. It’d be fun to chat. 👇👇👇

It would mean a ton if you would share this article on TwitterFacebook, and LinkedIn!

#marketresearch #userexperience

Part 6 of 10: Tips to Writing The Perfect Survey

Lessons Learned from Programming 2,000 Surveys Over 20 Years

You can find the start of this series here. As you are going through this article, please 👍 and share if you find things useful. 

Tip 6: Screener: Quotas/Terms

What is a Screener? This is the part of the survey that screens respondents ensuring you are talking to the right person.

If you can define who you want to talk to then you can put everyone else in a bucket and terminate them!

No alt text provided for this image

Why would you want to terminate people? The most obvious reasons are because talking to your mom about Snapchat is dumb and, God forbid, if someone looks at the data that includes your mom’s view of Snapchat you are screwed.

This is an example of a screening question…

S1. What is your age? 

13–19

20–24

25–29

30–34

35–39

Your Mom @term

If you only want to talk to a certain type of person, be darn sure that is the only person in your sample.

Quotas are the final piece in your Screener. If there are several types of buyers then you’ll want to have quotas. In my above example, I may want to have the following quotas to ensure my sample is representative of the general population

Quota:

13–19 N=50

20–29 N=50

30–39 N=50

Quotas can get pretty crazy. Let me know if you have specific questions about them. Here is a more detailed overview.

CONCLUSION

If you are interested in my next project, you can subscribe at begreat.io. As always, I hope you found this content useful and wish you only the best. If you’d like to add to the conversation…please drop a note below. It’d be fun to chat. 👇👇👇

It would mean a ton if you would share this article on TwitterFacebook, and LinkedIn!

#marketresearch #userexperience

Part 5 of 10: Tips to Writing The Perfect Survey

Lessons Learned from Programming 2,000 Surveys Over 20 Years

You can find the start of this series here. As you are going through this article, please 👍 and share if you find things useful. 

Tip 5: Sample

Sample is the profile of your respondents and is often times referred to as “Sample Frame”:

Sample Frame                          N=200

 Male/Female                          50%/50%

 Have visited a bar in last 2 months  100%

 Use phone while at bar               N=80

Let’s decode our sample frame…

  1. Sample Frame N=200 means we are looking for a total of 200 respondents.
  2. Male/Female 50%/50% means we want an even split between gender.
  3. Have visited a bar in last 2 months 100% means that all qualified respondents are bar goers
  4. Use phone while at bar N=80 sets a minimum of 80 respondents who fit this criteria. Note, it is likely that the total pool of respondents who use a phone at bars is much higher…this is just setting a floor to ensure we have enough people in this bucket.

Have a minimum of 80 respondents in each segment you want to analyze. This gives you a big enough group to ensure you have a representative sample.

Pro Tip: Why percentage instead of set numbers? Because we may adjust our total N and this keeps things symmetrical.

CONCLUSION

If you are interested in my next project, you can subscribe at begreat.io. As always, I hope you found this content useful and wish you only the best. If you’d like to add to the conversation…please drop a note below. It’d be fun to chat. 👇👇👇

It would mean a ton if you would share this article on TwitterFacebook, and LinkedIn!

#marketresearch #userexperience

Part 4 of 10: Tips to Writing The Perfect Survey

Lessons Learned from Programming 2,000 Surveys Over 20 Years

You can find the start of this series here. As you are going through this article, please 👍 and share if you find things useful. 

Tip 4: Survey Structure

All surveys have the same basic structure:

  1. Screener
  2. Diagnostics
  3. Profile

Sketch out your survey structure so that you know where to put certain types of questions.

How do you know what type of questions should be put together? I’ll cover this in a bit.

CONCLUSION

If you are interested in my next project, you can subscribe at begreat.io. As always, I hope you found this content useful and wish you only the best. If you’d like to add to the conversation…please drop a note below. It’d be fun to chat. 👇👇👇

It would mean a ton if you would share this article on TwitterFacebook, and LinkedIn!

#marketresearch #userexperience

Part 3 of 10: Tips to Writing The Perfect Survey

Lessons Learned from Programming 2,000 Surveys Over 20 Years

You can find the start of this series here. As you are going through this article, please 👍 and share if you find things useful. 

Tip 3: Analytics Plan

Write your presentation prior to going live with your survey. 

You are going to have to do this work regardless, so it isn’t a waste. 🙂

By writing your presentation ahead of time you are forced to think through the practical implications of answering your objective which will help ensure you have the right profile, diagnostic, etc. questions. 

Here is an example from the survey I referenced above. I’m citing a few placeholder (###) stats which my research will need to address.

No alt text provided for this image

Additional Benefit: By creating this slide I recalled similar work I’ve done for Pepsi and Coke and realized we should measure drink purchases and length of stay.

CONCLUSION

If you are interested in my next project, you can subscribe at begreat.io. As always, I hope you found this content useful and wish you only the best. If you’d like to add to the conversation…please drop a note below. It’d be fun to chat. 👇👇👇

It would mean a ton if you would share this article on TwitterFacebook, and LinkedIn!

#marketresearch #userexperience

Part 2 of 10: Tips to Writing The Perfect Survey

Lessons Learned from Programming 2,000 Surveys Over 20 Years

You can find the start of this series here. As you are going through this article, please 👍 and share if you find things useful. 

Tip 2: Stay on Point & Keep it Short

You may have heard the adage,

“Every dog has to pee on the tree.” 

I’ve personally programmed well over 2,000 surveys and authored at least a quarter of that. The single biggest and most common problem with surveys is scope creep.

Once you finish writing your survey you usually show it to stakeholders who, inevitably, have edits and additions. These additions are hard to push back on because, after all…

adding questions gives you additional data. 

But remember, a human being is on the other end of this survey. Taking a survey takes energy, focus and discipline.

Just think about the last long survey you took. At about 4 minutes, respondents start caring less about giving you correct answers and more about just finishing the survey. I’ve done tests on this with top brands where I take known transactional data like last item purchased and ask respondents,

What was the last item you purchased? 

Comparing stated response to their actual behavior we see respondents’ recall accuracy is over 90% at the start of a survey but falls to nearly 60% after minute 6.

So, how can you ensure your survey doesn’t suffer from question bloat? When writing your survey…

include the objective

 at the top of your survey. 

Below is an example of a survey I created yesterday…

Owner Survey Objective: Identify optimal product price for units and revenue

This creates clarity for you and your stakeholders. Every question asked will be viewed through this lens. If it doesn’t inform the objective then ruthlessly cut it. Remember, “nice to know” will compromise the quality of your findings.

CONCLUSION

If you are interested in my next project, you can subscribe at begreat.io. As always, I hope you found this content useful and wish you only the best. If you’d like to add to the conversation…please drop a note below. It’d be fun to chat. 👇👇👇

It would mean a ton if you would share this article on TwitterFacebook, and LinkedIn!

#marketresearch #userexperience

Part 1 of 10: Tips to Writing The Perfect Survey

Lessons Learned from Programming 2,000 Surveys Over 20 Years

After 2 decades in the consumer research space, I forget that writing a survey can be daunting for the first-timer. Over the next 10 days I’ll be writing 10 posts about survey writing best practices for market research and user experience pros. Each post will have one tip which will elevate your survey game regardless if you are writing your first, or your thousandth survey and ensure you get the best possible answers.

As you are going through this post, please 👍 or share if you find things useful. 😃Thanks!

Tip 1: Question Numbering

We think of surveys as this neat linear flow of questions and answers. Sadly, that simply isn’t the case. Writing a questionnaire is a very messy process…especially if it involves getting feedback from others.

Instead of using “question numbers” use “question labels”. Why? Because each question is a variable in your dataset and variables have labels. So, why is that important? Because, in the same way that during the survey writing phase you’ll want to reference your questions with their numbers, you’ll want to reference variables with their labels throughout the analysis and presentation phases. This way, your question label can be consistency referenced during the life of your project.

I use a basic naming convention where each question is referenced more like a variable. For example,

S1. What is your gender?

  • Male
  • Female

The “S1” is the label of the question and will stay with the question regardless of its location in the survey, e.g. it may get moved, removed and added back many times during the process. This permanent reference keeps data straight as well as removes confusion when people are referencing specific questions.

Always use a letter for the first character of your question label. You can use the same letter for questions that are similar in nature, like demographic questions start with the letter “D”, psychographic questions with “P”, screening questions with “S”, etc. This gives you a cheat sheet when looking at your survey and ties your data back to your questionnaire.

PRO TIP: If you label questions that you frequently use across other projects such as…

“What is your gender?” 

…with the same label, e.g. reserve S1 to always be your gender question OR simply label that question “gender“, you create comparability across projects giving your one-off research a long tail of value for research on research. This is especially helpful if you use a tool like KnowledgeHound for data visibility and accessibility over time.

..with the same label, e.g. reserve S1 to always be your gender question OR simply label that question “gender“, you create comparability across projects giving your one-off research a long tail of value for research on research. This is especially helpful if you use a tool like KnowledgeHound for data visibility and accessibility over time.

CONCLUSION

If you are interested in my next project, you can subscribe at begreat.io. As always, I hope you found this content useful and wish you only the best. If you’d like to add to the conversation…please drop a note below. It’d be fun to chat.

It would mean a ton if you would share this article on TwitterFacebook, and LinkedIn!

#marketresearch #userexperience