How to avoid biases in your questionnaires?

A well-constructed questionnaire is one that allows you to gather the most authentic, reliable, and reality-based responses from your respondents. It is therefore extremely important that you eliminate any risk of biasing their answers.

💡 A bias is an element that systematically disturbs the authenticity of the interaction (direct or indirect) between the respondent and the person seeking information.

Here are some simple and effective ways to avoid bias in your surveys:

• Avoid yes/no questions
• Do not ask leading questions
• Pay attention to the order of questions and answers
• Prioritize anonymity whenever possible
• Clarify your language
• Eliminate assumptions
• Sometimes prefer open-ended questions

Avoid yes/no questions

Try to avoid questions that can be answered with yes or no. Indeed, respondents tend to want to say yes to appear positive and ambitious. This is known as the acquiescence bias.

Rephrase your questions. For example, fora concept test, instead of "Do you find it essential to have nice packaging when you buy cookies?", rather ask: "Which of the following do you find essential when purchasing cookies?".

Do not ask leading questions

Be cautious of leading questions that push respondents to answer as you wish. This sometimes results in aconfirmation bias.. You can remedy this by avoiding unnecessary adjectives in your questionnaire. For example, don't say "What do you think of the delicious chocolate [brand]?". Instead, say "What do you think of the chocolate [brand]?".

Don't forget that responses to scale questions can also be suggestive. Ensure that your responses to scale questions include as many positive options as negative ones, and especially not just positive options.

Pay attention to the order

The way you order your questions and responses affects how people react. This is known as the order bias.

In questions

Order bias can occur when a question you've asked or an item you've presented impacts how your respondents perceive the rest of the survey. For example, if you present a series of advertisements, all subsequent advertisements will likely be compared to the previous ones in the respondent's mind, even unconsciously. Here’s what you can do to avoid order bias in questions:

• Don't give the answers to questions you’re going to ask along with those that come before. Think about the order in which you ask the questions to ensure you progress from the most general at the beginning to the most specific towards the end of the survey.
• Whenever possible, ask the questions randomly. The Episto platform includes this function and allows you to shuffle the order of the questions in your questionnaire.

In responses

The responses given can also be influenced by the order in which you present them. People are more likely to choose the answer(s) at the top of the list. If this happens, you won’t get an accurate representation of your audience's opinions. Episto's programming platform will allow you to randomize the response options. It's wise to do so for all answers to all questions, except when using scale responses.

Prioritize anonymity whenever possible

Don't disclose your identity (brand)! If you do, you risk causing a sponsorship bias, meaning that respondents may feel compelled to give positive feedback about the brand asking the questions. Moreover, it prevents respondents from allowing their perceptions of your brand to cloud their responses.

You generally want to know what consumers think of your brand, but sometimes you want to know what people think independently of your image. For example, if you are conducting research on new product development, you want to know consumer opinions without your brand's image, be it positive or negative, influencing the responses given.

During recruitment campaigns on social media, Episto systematically ensures not to cite or mention the brand to avoid this sponsorship bias.

Clarify your language

Avoid confusing respondents or leaving room for interpretation in the responses by being objective in your language. For example, instead of using terms like "recently," "often," or "rarely," specify the time period as "daily," "weekly," or "once a year." Make sure you know how your ideal customer speaks, which will help you tailor your language.

There are 6 levels of proficiency in the French language from A1 (beginner level) to C2 (perfectly correct expression on all subjects). The plain language corresponds to the B1 level, which is understood and mastered by 95% of the population. Therefore, it is advisable to use this level of language in your questionnaires to be understood by all your respondents. Also, don't forget to limit anglicisms and explain all technical terms that you will need to use in your questionnaire.

‍

Eliminate assumptions

Avoid making assumptions that force respondents to answer dishonestly. You can easily remedy this by asking yourself whether you should include the options "None," "Not applicable," and "Other" in your list of responses. For example, the question "What type of laptop do you own?" assumes that the respondent owns a laptop; you should ensure they can answer honestly by adding the options "None" or "Not applicable."

Instead of "What did you enjoy about your last vacation?", rephrase the question into two distinct parts: "What did you think of your last vacation?" "Very pleasant," "Fairly pleasant," etc. Don’t forget to add an option for the response "I did not take a vacation [in the past year]," then follow this question with an open question like "Why do you think that?" With Episto, it is even possible to customize the wording of a question based on the answer given to the previous question.

Sometimes prefer open questions

Favor open questions if you're not sure you can provide an exhaustive list of responses. In this case, to avoid any bias, you would do better to analyze written answers rather than constrain responses into an exclusive choice.

For instance, when you propose brand names from a specific sector, it's not complicated to provide respondents with a simple exhaustive list of brand names. However, when it comes to reasons why respondents wear makeup or go on vacation, some answers may be unexpected but no less interesting!

Thanks to the smart features of the Episto platform, it will be easy for you to analyze these open responses and take into account unexpected answers.

Do you want to build a questionnaire suitable for social media studies that truly engages your respondents? Check out this article to discover Episto's tips.