Tool 6: Putting together a Questionnaire

Once you have identified the people you need to engage with, you can start thinking about what questions to ask. There are a few key points:

• Think clearly about your aims for engaging people – why are you doing it? What decisions will it influence?
• Don’t just ask about things that you think are appropriate – think from the customer’s point of view. What would they want to tell you about?
• Value for money means we have to challenge the way the service is now and question if the need could be met more effectively. Use public engagement to explore this key issue.
• Ideally, you should engage people on which questions to ask

 There are two types of questions that can be asked open questions and closed questions.

• Open format questions are those that ask for unprompted opinions. In other words, there are no predetermined set of responses, and the participant is free to answer however they choose.
• Closed format questions usually take the form of a multiple-choice question. They are easy for the respondent and are easier to be analysed once the questionnaires are returned. Closed format questions also make it easier to track opinion over time by administering the same questionnaire to different but similar participant groups at regular intervals.   Key things to remember when designing questions is whether they are open or closed format

Clarity

Questions must be clear, succinct, and unambiguous. The goal is to eliminate the chance that the question will mean different things to different people. It is best to phrase your questions empirically if possible and to avoid the use of necessary adjectives. For example, if asking a question about frequency, rather than supplying choices that are open to interpretation such as:

• Very often
• Often
• Sometimes
• Never

It is better to quantify the choices, such as:

•  Every Day or More
• 2-6 Times a Week
• About Once a Week
• About Once a Month
• Never

Leading Questions

A leading question is one that forces or implies a certain answer. It is easy to make this mistake in formulating the question as well as the choice of answers. A closed format question must supply answers that not only cover the whole range of possible responses, but that are also equally distributed throughout the range. All answers should be equally likely. 

Phrasing

Most adjectives, verbs, and nouns in English have either a positive or negative connotation. Two words may have equivalent meaning, yet one may be a compliment and the other an insult - for example "child-like" and "childish", which have virtually identical meaning. Child-like is an affectionate term that can be applied to both men and women, and young and old, yet no one wishes to be thought of as childish.  

Embarrassing Questions

Embarrassing questions dealing with personal or private matters should be avoided - unless it is about the topic of the consultation; in that case you have to emphasise the anonymity of the data. Your data is only as good as the trust and care that your respondents give you. If you make them feel uncomfortable, you will lose their trust.

Prestige Bias

Prestige bias is the tendency for respondents to answer in a way that make them feel better. People may not lie directly, but may try to put a better light on themselves.

Questionnaire length

There is no consensus on how many questions should be asked but an overly long questionnaire will lose the respondents attention and may result in a lower response rate. If your survey is over a few pages, try to remove questions. For each question ask, "How am I going to use this information?" If the information will be used in a decision-making process, then keep the question... it's important. If not, remove it. 

Things to think about

• Formulate a plan for doing the statistical analysis during the design stage of the project. Know how every response will be analysed and be prepared to handle missing data. If you cannot specify how you intend to analyse a response or use the information, do not use it in the survey.
• Provide a well-written cover letter. The respondent's first impression comes from the cover letter. It provides your best chance to persuade the respondent to complete the survey. 
• Include clear and concise instructions on how to complete the questionnaire. These must be very easy to understand, so use short sentences and basic vocabulary.
• Be sure to print the return address on the questionnaire itself (since questionnaires often get separated from the reply envelopes). 
• Begin with a few non-threatening and interesting items. If the first items are too threatening or "boring", there is little chance that the person will complete the questionnaire. People generally look at the first few questions before deciding whether or not to continue. Make them want to continue by putting interesting questions first. 
• The wording of a question should be simple and to the point. Do not use uncommon words or long sentences.  
• Leave adequate space for respondents to make comments. One criticism of questionnaires is their inability to retain the "flavour" of a response. Leaving space for comments will provide valuable information not captured by the response categories. Leaving white space also makes the questionnaire look easier and this increases response levels. 
• Place the most important items in the first half of the questionnaire. Respondents often send back partially completed questionnaires. By putting the most important items near the beginning, the partially completed questionnaires will still contain important information.
• When asking for personal information such as age, location, gender and any other demographic information, ask only what you need to, and think about how you will anlayse the data so that you have the information you need. Do you need to know what different age groups in a particular area think? Ideally this information should be asked towards the end of the questionnaire, so that respondents are not put off early on by personal questions.
• It is important to group items into coherent categories. All items should flow smoothly from one to the next. 
• If a questionnaire is more than a few pages long, include some identifying data on each page (such as a respondent ID number).
• Make it convenient. The easier it is for the respondent to complete the questionnaire the better. Always include a self-addressed postage-paid envelope.

A national database of questions can be found in the: ESRC Question Bank

Further information on Surveys can be found at: I&DeA Customer Insight Protocol

Authors: Essex County Council, Rochford District Council and Uttlesford District Council. You are free to share, copy or adapt this work but you must attribute it to EngageEastEngland.

Site design by Cleveratom Limited

Images: Mass Ascension by a4gpa (Attribution-Share Alike 2.0 Generic Creative Commons Licence), Pink Penstemons by aussiegall (Attribution 2.0 Generic Creative Commons Licence), Tools In The Garage (1) by Viewoftheworld (Attribution 2.0 Generic Creative Commons Licence.)