Survey design: writing questions people actually answer

One idea per question

Double-barrelled questions ('Was support fast and friendly?') produce unusable data, because a respondent who found support fast but unfriendly has no honest answer. Split them into two questions.

Watch for hidden double-barrels in the response options too: an option like "Affordable and reliable" forces two judgements into one click. Each option should express exactly one idea.

Scales

Use a consistent direction and labelled endpoints. A Likert scale with five points is plenty for most attitudes; reserve the 0-10 range for NPS so the two are not confused.

Decide deliberately about a neutral midpoint. An odd number of points offers one and is fairer when people genuinely have no view; an even number forces a lean and is better when fence-sitting is unhelpful - but it frustrates the genuinely undecided, so choose per question.

A matrix question is efficient but fatigues respondents and invites straight-lining. Cap rows at roughly five to seven, keep columns short, and break long grids across pages in a multi-page form.

Choosing the right question type

Match the type to the decision the data will inform. If you need to know which thing matters most, a ranking question forces the trade-off that a grid of independent ratings hides. If the choice is inherently visual, a picture choice preserves information a text label would lose.

Reach for the right metric, not the familiar one: CSAT for satisfaction with a specific interaction, CES when the question is how hard something was, NPS for broad loyalty. Putting all three in one survey is fine, but mind the order effects below.

Order effects

Earlier questions prime later ones. Ask sensitive or open questions after rapport-building ones, and put your priority metric - often NPS - early, before opinions drift in response to everything else you asked.

Beware contrast and assimilation: asking about a specific bad experience just before a general satisfaction question drags the general score down. Randomising option order within a question reduces primacy bias for long choice lists.

Who you don't hear from

The most dangerous error in survey work is not a badly worded question, it is survey sampling bias - the people who answer differing systematically from the population you care about. A long survey amplifies it by driving off everyone but the most motivated.

Defend against it by keeping the survey short, distributing through channels that reach the whole population rather than only your happiest customers, and explicitly noting who is missing when you report results. A representative small sample beats a large skewed one every time.