Research methods

Questionnaire design: the mistakes that cost you the answer

Leading questions, double-barrelled items, scales without a midpoint, and the question that reads neutrally in English and as an accusation in Arabic.

A questionnaire is an instrument, and like any instrument it can be built in a way that guarantees a particular reading. Most of the ways this happens are well documented and still routine.

Leading questions

"How satisfied were you with our excellent service?" is an obvious example and rarely survives review. "Did our team resolve your issue quickly?" is the same fault in a form that usually does survive — it presupposes resolution and it supplies the adjective.

The repair is to ask what happened before asking how it felt. "Was your issue resolved?" then "how long did it take?" then "how satisfied were you with how it was handled?" Three questions, three findings, no presupposition.

Double-barrelled items

"Were the staff friendly and knowledgeable?" cannot be answered by a respondent who found them friendly and uninformed, which is the most common combination in retail. Whatever they choose, the answer is wrong and you cannot tell which half it refers to. Split every item containing "and".

Scales

Use one scale throughout. A questionnaire that switches between five-point and seven-point scales, or between agreement and satisfaction wording, forces the respondent to re-learn the task and produces noise that looks like variance.

Label every point, not just the ends. Numbered scales with only the extremes labelled are interpreted differently by different respondents, and differently again across languages.

Whether to include a midpoint is a real decision rather than a default. A midpoint captures genuine neutrality and also collects everyone who did not want to think about it. If the decision the survey informs needs a direction, a forced choice is defensible — but say in the report that it was forced.

Order effects

The first question sets the frame for everything after it. Asking about a specific irritation before asking for an overall rating drags the overall rating down. Ask the general question first, then the specifics — and if you must ask both, rotate the order across the sample and check whether it mattered.

Questions people cannot answer

"How likely would you be to buy this at 15 per cent higher price?" asks for a prediction about a hypothetical. People answer it, which is the problem. The response is a stated attitude and it is routinely reported as a demand forecast.

Ask about behaviour that has happened. Where you must ask about intent, report it as intent.

Demographics: only what you will use

Every demographic question costs completion rate, and several cost trust. Ask for what your analysis plan actually cuts by, offer "prefer not to say" on anything sensitive, put them at the end, and never pre-select a default. A pre-selected default guarantees false data on every unedited submission — which is a fault we have seen shipped on live forms more than once, including on the site this one replaces.

Pilot it, then cut it

Pilot on twenty real respondents and watch where they hesitate. Then cut every question you cannot name a decision for. A questionnaire is finished when removing one more question would cost you an answer you need, not when it covers everything anyone asked for. The same discipline governs a satisfaction survey and a face-to-face interview guide.

Next step

How we design an instrument, in full.

Scoping, scorecard design, shopper vetting, fieldwork, QA, reporting and data protection — the seven stages, and what each one is for.

Read the methodology Number unconfirmed +974 5205 2051 2026 company profile +974 7743 4330 live site

Sun–Thu, 08:00–18:00 AST