Idea validation survey questions that don't just collect yeses

A survey is a blunter instrument than a live interview. You lose the improvised follow-up, the tone of voice, and the chance to notice hesitation. In exchange you get scale and consistency. That trade only pays off if the questions hold up against the two things that ruin survey data: acquiescence bias (people agreeing by default) and hypothetical answers.

Here's how to write idea validation survey questions that hold up, plus a bank you can adapt.

Structure the survey around your hypotheses

Before writing a single question, list the claims you're testing: problem, willingness to pay, behavior, solution fit, market. Every question should map to one of them. If a question doesn't test a hypothesis, cut it, because it only makes the survey longer and lowers completion. If you don't have hypotheses yet, start with how to write a startup hypothesis.

Open with a screener

The first question should qualify or disqualify. Ask "In the last 30 days, how many times have you [done the core activity]?" with an option for "never," and route "never" straight to the end. Every unqualified response dilutes your signal, and in a survey you can't tell who's who later.

Problem questions

Ask about frequency and cost. How strongly people say they feel about it matters less.

• "When did you last deal with [problem]?" (Within the last week / month / 3 months / longer / never)
• "How much time did it take you the last time?" (open number)
• "What did you do about it?" (open text; this one is worth the extra length)
• "How much of a problem is this for you right now?" (5-point scale, but treat it as weak evidence next to the behavioral answers)

Workaround and current spend

• "What do you use to handle this today?" (multi-select with an "other" and a "nothing" option)
• "Roughly what do you spend on this per month, counting tools and your own time?" (bands)
• "Have you paid for a tool to help with this before?" (yes/no; if yes, "which, and do you still use it?")

Willingness to pay

Use a real anchor instead of an open "how much would you pay." The van Westendorp price-sensitivity questions work well in surveys:

• "At what monthly price would this be so expensive you wouldn't consider it?"
• "At what price would you think it's expensive but still consider it?"
• "At what price is it a bargain?"
• "At what price is it so cheap you'd question the quality?"

There's more on this in willingness to pay survey questions.

Behavior and adoption

• "Have you switched tools for this in the last year?" (yes/no, then "what prompted it?")
• "What would have to be true for you to change your current approach?" (open text)
• "Who decides whether your team adopts a new tool like this?" (self / manager / committee / other)

Solution fit (only after the problem is established)

Show a one-sentence description, then ask:

• "How well does that match a problem you actually have?" (not a match / partial / strong)
• "What's missing or wrong about it for your situation?" (open text)
• "What do you currently use that's closest to this?"

Question-writing rules

• No leading language. "How frustrating is the slow export?" assumes it's slow and frustrating.
• Behavior over intention. "Have you done X" beats "would you do X."
• One idea per question. Nobody can answer "Is it slow and hard to use?" cleanly.
• Keep it under 12 questions. Completion drops fast after that, and the people who drop off aren't random.
• Put the open-text questions early. People stop typing full answers as they get tired.

Scoring the results

For each hypothesis, decide in advance what counts as confirmed. For example: "Problem confirmed if 60% or more report dealing with it in the last month AND at least a third describe an active workaround." Setting the bar before you see the data keeps you from talking up a weak result. See when to kill a startup idea.

Common questions

How many responses does an idea validation survey need?

For a directional read, 30 to 50 qualified responses. You're looking for clear majorities and strong patterns. Statistical significance isn't the goal at this stage.

Survey or interviews first?

Interviews first if you're still learning the shape of the problem, because you don't yet know what to ask. Switch to a survey once you can write precise questions and want to check whether the pattern holds at scale.

How do I stop people from just clicking through?

Use a screener that ends the survey for people who don't qualify, add an attention-check question, and keep it short. Throw out responses that fail the attention check or finish implausibly fast.

Can I use a free survey tool?

Yes, for the mechanics. The hard part is writing unbiased questions and setting your scoring bar honestly, and no tool does that for you by default.

Let the survey write itself

Hypothis builds a screened idea-validation survey from your hypotheses, keeps the questions Mom-Test-disciplined, adds adaptive follow-ups, and scores each hypothesis against the responses using the bar you set. It's free during early access. See how it works.