In this introduction to a 10-part Fundamentals of Research blog series, Derek Jones, Founder of D&M Research, talks about all the steps you need to know to be an effective practitioner, manager or buyer of marketing research in today’s data-led world.
The series re-focuses on the fundamentals which Derek argues are either not understood or are being overlooked in the favour of frugality and agility, leading to often erroneous decisions being made off the back of spurious data. Join us as we guide you through everything from spending more time planning at base camp, to asking the right questions of the right people, to analysing and making sense of your data for the purposes of brilliant decision making. At the end of the series, you will be able to download the Research Playbook. Enjoy.
Tools such as Survey Monkey, Zoomerang, Survey Gizmo or Quick Survey are freely available and making research far more accessible to smaller and medium enterprises, than ever before.
However, just having a hammer and a saw does not make you a carpenter, and along with the increasing number of organisations currently doing surveys comes the risk of poorly designed and executed surveys. Now, given the main reason to do research is to inform decision making, the quality of your decision making can be impacted by the quality of your research and, in particular, your survey questionnaire.
So the team at D&M Research thought it was time to give some broad tips on how to ensure you not only ask the right questions, but also ask these questions the right way.
Below are our top 10 expert tips to put you on track to brilliant research, and we will be expanding on these in much more detail in upcoming blog posts!
It’s imperative you spend some time at base camp before setting off. Think long and hard about your survey, who are you targeting? How will you get them and how will you ensure they are representative of the target audience? What issues do you anticipate? How long will the survey be? Think about these types of questions and try to write down on one page a research plan.
2. Goals and Objectives
Research usually starts with a business problem – try and define this in a sentence or two (what is the reason for doing this research?). Once you have done this, try to define the decisions you will make at the end of the research. This is a really neat way to focus your goals and objectives. Finally, translate this into an overall objective and supporting objectives to get the job done. This will then be your master plan for the creation of your survey.
3. Design and Structure
Before you put pen to paper, start to plan out your design. Think in terms of broad topics or sections. Usually you will need to start with some sort of screening questions to ensure you are talking to the right people, you might want to mask this question by giving multiple options to reduce self-selection. Then introduce the subject or category from a broad perspective narrowing down into specifics as you go. Finally, finish with a classification section and always ask sensitive questions like income at the end.
4. Survey Length
Survey length is one of the most important considerations in quality research – generally the shorter the better, but you have also got to balance this with meeting your objectives. Try to keep your surveys to 5-15 minutes, and offer incentives for longer surveys. One way of doing this is to ensure that each question is matched to a specific objective. Think of it as each question earning the right to be there. If it’s a nice to know and you don’t have room, it's out! It’s a good idea also to tell respondents realistically how long the survey will take.
5. Minimise Bias
Bias is the most common cause of poor research. Although eliminating bias is almost impossible, it is important to plan strategies to minimise it at all costs. Some of the key strategies to consider are: always have balanced scales (agree or disagree), don’t introduce topics that will affect later questions (plan an order), don’t use double-barrelled questions, match your questions to the responses you offer, include randomisations of answers where possible, and always provide clear instructions to respondents.
6. Question Routing
Make sure you define who answers each question clearly and use skip logic to ensure that the respondent only sees the relevant questions. Again, go back to your plan and think about how this will work up front, keeping relevant sections and questions together.
7. Timing and Testing
Once you have programmed your questionnaire, take the time to thoroughly test it including all the skip logic. Pilot it on your colleagues and ask for their candid feedback – was there anything they didn’t understand or wasn’t clear? When you finally launch your survey, check the data after the first day to ensure that all the right questions are been appropriately completed.
8. Sample Size and Quotas
Although it might be tempting to get as many responses as possible, it’s important to control the sample you end up with by setting up quotas. There is no point having 70% females in your survey if you are doing a population survey where the actual proportion is closer to 50%. This will introduce a bias to your results. Sample size is important too, and as a rule of thumb, sample sizes of over 1,000 will generally not increase your accuracy, so spend more time getting the right sample than the biggest sample. Use reminders to get at harder to achieve sub-samples like younger males.
9. Data Cleaning
It’s really important that you also clean your data when your survey is finished. Look for responses that are inconsistent, contain "rubbish" answers to open-ended questions, or were completed too quickly (they might just be clicking anything to complete). It’s a good idea to over-sample by 10% to allow for the removal of some respondents.
10. Data Analysis and Reporting
This is perhaps the most specialised skill required in the survey process. If you are not comfortable with how to process the data, then perhaps think about investing a small sum into professional help from the team at D&M Research. There is a lot to consider including filtering questions, bases, weighting, creating statistics, analysis banners and statistical testing to discover real differences. Most of the DIY packages offer some sort of basic analysis tools, but be careful with your analysis and check it thoroughly to ensure it is correct, before you start drawing conclusions and writing a report.
Feeling overwhelmed? D&M Research is here to help you with any or all of these areas. We offer affordable consulting packages for survey preparation and analysis, all the way up to comprehensive full-service research solutions. Ensure your next research project is a brilliant success by giving us a call or email today.