This week a fundraising advisor described a founder who wrote 18 interview questions covering pricing, features, workflow, distribution, pain points, competitors and half the product roadmap. The interviews happened. Lots of interesting things got said. Afterwards, nobody knew what to do with any of it. If that stings, it is because almost every founder has run that exact study.
Why do most customer interviews produce nothing?
Because they were never designed to settle anything. Research that starts from I want to understand the market has no failure condition. Whatever people say becomes a note in a document, and the document becomes a prop. The founder feels diligent, the roadmap stays exactly where it was, and the pile of interesting anecdotes is deep enough to defend any decision anyone already wanted to make.
The same week, another founder admitted he had agreed with a famous customer interview guide for years and never once followed it, until someone tore his plan apart live. His core assumption was simply wrong. Not slightly off. Wrong in a way twenty minutes with real customers would have exposed.
What separates evidence from anecdotes?
One thing: a decision named before the data arrives. Evidence is any observation that could change what you build next. An anecdote is an observation collected without a question it could answer. The material is identical. The difference is whether you decided, in advance, what would have to be true for you to continue, narrow, change or stop.
This is why the question what evidence would change our decision is worth more than any interview script. It converts research from a comfort ritual into a measurement. It also makes fooling yourself harder, because you wrote down the losing condition while you were still objective.
Why do founders avoid decision-first research?
Because a roadmap does not argue back and a stranger might. Naming the decision upfront means accepting that the answer might be stop. Most of us would rather run a study that cannot say stop. That is not a research problem, it is a courage problem wearing a research costume.
What does real evidence look like?
It looks like behavior from people who were free to reject you. What someone did, paid, cancelled, came back to or quietly abandoned. Stated interest from friends, upvotes from strangers and polite yeses in interviews all share one flaw: nobody had to give anything up. A signal that costs the other person nothing measures your likability, not your demand.
There is a growing club of founders who shipped real products and heard silence. Every one of them had encouraging conversations first. The silence was not a marketing failure. It was the first moment their idea met people who were allowed to say no.
Key takeaways
- Research counts as validation only when it is built to settle a decision you named in advance.
- An 18 question interview guide is usually a way to postpone a decision, not inform it.
- Write down the losing condition before you collect a single answer.
- Trust what people did and paid over anything they said they might do.
If your research cannot say stop, it is not research. It is a longer way of saying yes to yourself.