Scaling Qualitative Research with AI

Qualitative research often loses time to logistics. Recruit eight to twelve people, schedule calls, transcribe them, and compare notes. By the time the themes are clear, the product decision may already be made.

AI interviews offer a way to collect more conversations without scheduling every one. Anthropic’s study of 1,250 professionals is a useful example of what that approach can reveal and where it falls short.

What 1,250 interviews revealed

The study included 1,000 people from the general workforce, 125 creatives, and 125 scientists. In the general sample, 86% reported that AI saved time and 65% were satisfied with its role in their work. Yet 69% mentioned social stigma around using it. One fact-checker described keeping quiet when a colleague criticized AI.

That tension is more useful to a product team than a satisfaction score alone. Someone may find a tool valuable while avoiding any visible sign that they use it.

The report also compared interview responses with a separate analysis of Claude conversations. Respondents described their use as 65% augmentation and 35% automation; the conversation analysis found 47% augmentation and 49% automation. These were different samples and methods. The gap suggests questions worth investigating, not proof that individual respondents misrepresented their behavior.

Scientists raised the cost of checking answers. Creatives emphasized retaining control. Both point to a product question: how much useful work remains after the user verifies and adapts the output?

How the interviews worked

Anthropic divided the process into planning, interviewing, and analysis. Researchers helped define the interview plan and reviewed the results. The AI conducted text conversations, asking follow-up questions within that structure.

This keeps a common line of inquiry across interviews while allowing a participant to explain an unexpected answer. It also produces transcripts that can be compared without a separate transcription step.

Human review still matters. A consistent interviewer can ask a consistently poor question. A convincing thematic summary can overlook a contradictory response. Researchers need to inspect the underlying conversations and decide what the evidence supports.

A version for small teams

I built a Quick-Chat Research Tool to explore a lighter version of this approach.

I would use it for a specific question where a survey feels too rigid but scheduling a full round of interviews would slow the decision:

• Early discovery: How do people handle the problem today, and where does the process break?
• Concept testing: What do they think a proposed feature does, and what would stop them from trying it?
• After launch: What prompted them to use or abandon the feature?
• Following up on analytics: What happened around a drop-off or unexpected usage pattern?

Keep the question narrow. Review a few transcripts before expanding recruitment. Check that the follow-ups invite concrete examples instead of nudging people toward the answer you hoped to hear.

Other tools to consider

Outset and Maze are examples of research platforms to evaluate if you need more than a small experiment. Looppanel is relevant when organizing and analyzing existing interviews is the main problem.

Choose based on the work you need help with: recruiting, conducting interviews, or analyzing evidence. Check data handling and transcript access as carefully as the generated summaries.

What text interviews miss

A text interview cannot show you a hesitation, a workaround on another screen, or the moment a participant gets stuck. People also describe their behavior imperfectly. Being interviewed by AI about AI may affect what they say, and a recruited sample may not represent your customers.

Use these interviews to find patterns and decide where to look more closely. Follow up on consequential findings with direct conversations or observation of the actual workflow. The benefit is getting useful evidence sooner, provided you remain clear about what you have not observed.

Research only helps if it changes a decision. Most Things You Could Build Aren’t Worth Building shows how a closer look at the work led me to reduce an agent’s scope.