November 17, 2015

The Art and Science Of Questionnaire Design

questionnaire design blog cover

Designing employee questionnaires is a tough thing to get right.

There really is an art and a science to it. The art, is in wording a question perfectly to get the most valuable response.

The science, is in understanding behavior to figure out things like how many questions to use, what words work better than others, and the balance between open and closed questions.

Considering how much time and energy is spent by companies to get this process right, it’s important to understand how to do it properly.

The biggest problem with questionnaire design is our unconscious bias.

It’s an even bigger problem than we realize, because as much as we try to remove biases from the process, they’re still there.

Two very prominent examples of this related to questionnaire design are:

  • “Yea-Saying”
  • Extreme responding

Yea-Saying is a type of bias where respondents to a survey have a tendency to agree with a question when in doubt.

An example of this is a question like “Our culture supports and trains their employees well”. If the employee doesn’t know whether it’s true or not, they’ll likely say yes to that question, because it sounds good if it’s true.

Extreme Responding is when there’s an opinion question from 1-5, and employees always choose either 1 or 5 (the most extreme responses).

This happens very often and is bad for reporting and analyzing the survey results.

To think that some companies will plan an entire year’s worth of initiatives based on faulty data is truly incredible.

Free Bonus: Download 40 must-have survey questions that you can use for your next survey.

How To Write Better Survey Questions

The best piece of advice I can give when it comes to questionnaire design is start with the goal in mind.

This is why smaller, more frequent surveys make so much more sense. You can have mini goals for each survey and design them much better.

Here are a few tips to help you write better survey questions.

  1. Balance Open And Closed Questions

    The easiest thing to do when designing a survey is to ask closed questions like multiple choice and opinion scales, but you’ll get an incredible amount of insight from open-ended questions that let employees speak their minds freely.

    The issue is with balancing the two.

    The best advice I can give here is to ease the employee into the open-ended question by asking a few closed ones first.

    Also, the more relevant the open-ended question is the more valuable their answer will be.

    This is exactly what we do in our employee survey software. We’ll ask contextual followup questions.

    As an example, we’ll first ask how likely they are to promote the company outside of work:

    Ambassadorship question in Officevibe

    And then immediately ask for qualitative feedback depending on their answer.

    ambassador followup

  2. Avoid Leading Words

    I’ve seen companies make this mistake many times.

    Leading words are words in the question that lead employees towards a certain answer.

    This is a great example of your unconscious bias at work.

    Consider this example:

    We recently updated our intranet to be easier to navigate and more user-friendly. What do you think of it?

    This example clearly shows that you want them to respond positively. You’re telling them that you think it’s easier to navigate and more user-friendly.

    Instead, keep your question neutral and simply ask “What are your thoughts on the new intranet?”

  3. Ask One Question At A Time

    It’s important to keep your employee survey questions very focused.

    It’s understandable that you want to know everything and you probably have a lot on your mind. Take the time to focus each question and make sure that every question has a clear goal.

    Consider this example:

    What did you think of the Christmas party? Is that one of our best perks? Did you enjoy yourself? Why or why not?

    Slow down, take a deep breath, keep your questions focused.

  4. Use Simple Language

    Imagine if you were taking a survey and saw something like this:

    The board deliberated on the validity of our A.T.L.A.S.T program through Q2 and found that there were practices that needed updating, do you agree?

    While this example might be silly and over exaggerated, it illustrates a good point.

    Use language that everyone understands and don’t assume that everyone knows the acronyms for your company programs.

Keep Your Surveys Short

SurveyMonkey analyzed how the length of a survey affects response rates and found some not-so-surprising results.

The more questions you ask, the less time your respondents spend, on average, answering each question.

One interesting finding was that respondents take more time per question when responding to shorter surveys compared to longer surveys.

This is important, because as employees go through your survey, the further along they go, the more they rush, the lower the quality/reliability of the answers become., an app that does Net Promoter Score surveys, analyzed 5 million of their surveys and found:

With each added click or question, your chances of someone opting out of a survey drop 50%

While that might not be the case for your surveys, it’s important to keep that stat in mind and keep your surveys short.

Survey eBook

Avoid these 15 common survey pitfalls and get tips from the experts.

Download now!

Sample Questions To Ask

Here are a few sample questions you can feel free to use in your next survey:

  1. Is there anything blocking you from getting your work done?
  2. Are we helping you advance your career at a pace you would like?
  3. What could we change about our team meetings to make them more effective?
  4. Are there any parts of our culture you’d like to change?
  5. What are your favorite parts about our culture?
  6. What could we do to help you be more productive?
  7. Are there any skills you would like to develop right now?
  8. Is there any part of your job that you want more help or coaching?
  9. Are you happy working here?
  10. What’s one thing we could do to help you enjoy your job more?
  11. How do you feel your work/life balance is right now?
  12. How do you feel about your compensation (salary and benefits)?
Free Bonus: Download 40 must-have survey questions that you can use for your next survey.

Any Questionnaire Design Advice For Us?

What are some ways you’ve run surveys in the past to get high quality answers? Let us know your thoughts below!


Read our latest content