Okay, so you’ve done your planning and set out your decision-centric goals and objectives. It’s time now to put pen to paper and start thinking about the research instrument – that is, the questionnaire.
But before you start just writing questions think about a design and structure for the instrument.
Good questionnaires have flow and take into consideration the respondent and the logical introduction of topics and specific areas of investigation. You also need to consider the impact of bias. Bias will be covered in more detail later but is also relevant to your design and structure as some questions may impact later responses. A great example is unaided brand awareness – there’s no point asking about prior brand purchase with a prompted list and then asking respondents which brands can they think of – why? It might seem obvious but you have just prompted them so your ‘top of mind’ responses will be biased.
So how do you create a design plan for a questionnaire? I like to think of a questionnaire as book with chapters, with each chapter addressing a specific topic. Plan your chapters to roughly address your objectives, although you may cover some objectives in more than one chapter, or a single objective with different questions in different chapters, either way its good practice to consider and plan these out.
But before you start planning out your chapters you have to think about whom you are going to let read your book. The very first thing you therefore need to consider is who is going to qualify for your survey – that is who is the target respondent? It’s pretty unusual that you want to speak to everyone, and at the same time it’s also pretty unusual to find a source of respondents that fits your target exactly – so what you need is to think about screening your respondents. Screening is the process whereby you screen in people who qualify for you survey and screen out people who don’t qualify. A simple example would be gender, age and area. For arguments sake let’s say I’m interested in speaking to females aged 25-54 who live in Sydney. I would need to ask these questions right at the front of my survey to make sure I’ve got the right people. It’s a waste of resources and respondents to do this at the end because you will end up throwing out those that don’t qualify. Screening usually goes beyond simple demographics and usually includes some sort of behavioural or category purchase questions. A good example might be being the grocery buyer and buyers in a particular category like frozen vegetables, so you’ll need to ask these questions up front as well.
It’s a really good idea to mask these types of questions to reduce self-selection and bias – especially if you are rewarding respondents who qualify. So using this example above:
S1. Which of these are you personally responsible for in the household you live in?
a) Doing the housework
b) Paying the bills
c) Buying the groceries
d) Maintaining a car
S2. Which of these have you purchased for this household in the last 6 months?
a) Frozen ice-cream
b) Frozen berries
c) Frozen vegetables
d) Frozen pre-made meals
This way a respondent doesn’t know the right answer and is therefore more likely to give truthful answers rather than answers that will get them into the survey and therefore access to the incentive. You would only then choose those who answered c) to both questions above to complete your survey thanking the rest for their time.
Okay, so now on to the book. First thing to consider is an introduction – you wouldn’t write a book without it so you need to start broad introducing the topic and then drill down to specifics, allowing logical breaks and change of topics that flow and fits a natural way of thinking.
A typical plan might look like this, again using the example above of frozen vegetables:
- Screening questions (key demographics and masked behaviour and/or category questions)
- Introduce the category (broad). Thinking now about frozen vegetables … ask broad questions like frequency of purchase and use, who consumes them, where & when purchased etc.
- This gets your respondent thinking about the category generally.
- Category Brands (specific) – Which brands can you think of? Which of these have you heard of before today? Which have you ever bought, currently buy, which is your main brand? etc.
- This gets your respondents now to think about the brands they know and use.
- Category Drivers (specific) – How important are each of these in which brand you choose?
- Now that they have thought about the category and the brands they are better positioned to think about what’s important.
- Brand Performance – How well do each of these brands deliver on these factors?
- Now that they have thought about brands and what’s important they can better assess the brands on what’s important.
- Demographics – Questions like marital and work status, education levels, household composition etc.
- It’s a really good idea to end your survey with demographics so you can classify your responses and look for difference between key groups. It’s it also a good place to ask any sensitive questions like income which might put people off earlier leading to a termination.
Once you have thought out your plan you have a blueprint for where your questions are going to go and how it will flow. This might change as you actually start to write your questions but is a really good practice to make the whole process easier and better for your respondent experience and data quality. Finally check your topics or chapters against your objective to make sure you have at least a place for them to be covered.
You are now ready to start drafting your questionnaire – good luck!