Qual-at-Scale: The Reality Behind the Hype

by Jeremy Cochran, PsyD

If you’ve been to an insights conference or read any industry news in the past six months, you will have undoubtedly heard (a lot) about Qual-at-Scale (QAS). QAS has quickly become one of the most talked-about methodologies in insights, promising the depth of qualitative at the speed and scale of quantitative. Over the last several years, startups have raised millions to develop QAS platforms. In a world where insights budgets need to go further and further, QAS is providing a viable option for many insights teams.

Yet even with all the noise, announcements, and promises that QAS providers have made, I’ve found, through discussions with clients and colleagues, that there are still many unknowns about QAS. How does it work? Where does it fit in the research toolkit? Does it deliver?

As Manager of Research & Development at Burke, part of my job is critically evaluating new technologies and frameworks in the insights industry to see what value they actually add to our work. As we have brought Qual-at-Scale into the fold of work we do for our clients, I’ve had the opportunity to assess it not just in theory, but in practice. Here is what I’ve learned that can help you understand if Qual-at-Scale is a good solution for you.

What exactly is Qual-at-Scale?

Qual-At-Scale is, essentially, what it says on the tin: a way of conducting qualitative interviews at a scale normally reserved for quantitative research. There are two primary elements that allow this to happen: AI moderation, or using AI to moderate multiple interviews at the same time, and AI Analysis, or using AI to quickly analyze hundreds or even thousands of interviews to form summaries, insights, and personas based on the interview data.

Combining these two elements, QAS allows you to get qualitative insights (themes, sentiment, emotions, etc.) faster and from a larger base size typically seen from traditional qualitative interviews.

Qual-at-Scale Provides Speed, Efficiency, and Breadth

Qual-at-Scale excels in providing an efficient way to obtain a moderate depth of insights from large groups of individuals. Studies that used to take weeks or months can now be done in days, with some caveats. In a time where insights professionals are asked to do more with less, this provides a strong option.

In addition, Qual-at-Scale can expand your research globally as most QAS platforms can quickly translate and conduct interviews in other languages. While this creates efficiencies, quality best practice is to still have native-language review to ensure any cultural context is conveyed appropriately.

Designing a Qual-at-Scale Study: Benefits and Watchouts

Overall, the Qual-at-Scale process is fairly simple: use the AI interface to design the interview, launch it through the platform or with your own sample, and use AI tools to analyze the results. Using the AI can make the process very speedy, but there are a couple of key watchouts that require a thoughtful approach.

Designing Interviews – Be Very Clear with Instructions to AI

The design phase is like traditional qualitative research where you identify research objectives and create a discussion guide. With Qual-at-Scale, however, the interview is divided into “base” questions (that the AI moderator will ask verbatim), and probing questions (AI asks during the interview based on your instructions).

While the AI interface helps you craft your discussion guide, my team found it to be a little over-ambitious at times. The AI would expand a small prompt into an entire study and suggest a lot of additions that you may not need. It’s helpful, but could rush you into feeling the survey is complete without fully thinking through what the interview needs to include.

In addition, we found that using the AI assistant is no substitute for having expertise in qualitative research and behavioral science in crafting research design. The AI does well at creating general discussions around a subject, but needs human guidance to apply frameworks and shape the discussion in the best way to meet business objectives. Knowing what kinds of answers you want and framing the interview questions around that is crucial for ensuring your research is actionable.

Finally, we also saw the importance of giving clear and thorough instructions on what to prompt on to the AI to ensure it asks good prompts. Since the probing questions are out of your direct control, it’s vital to give the platform as thorough of instructions as you can. Too broad or ambiguous guidance increases the risk of off-topic or irrelevant questions.

Sample Design – Quality and Feasibility Issues Still a Concern

One of the draws of Qual-at-Scale is being able to use quantitative sample providers, which are generally less expensive than qual providers. Using quant sample can broaden your potential respondent pool, hasten recruitment time, and shorten field, especially compared to traditional qual. We’ve found that most respondents that completed QAS interviews enjoyed the process and gave thoughtful responses.

However, a good percentage of respondents from quant panels abandoned the survey upon learning it would be a video interview, so comfort with AI moderation may be a concern and something we will continue to monitor.  In addition, the quality issues around quant sample, including fraud, inattention, and speeding, are still present. While requiring video is a quality measure itself, we’ve still seen low-effort respondents and mismatched targets. The QAS platforms have tools to catch some of these activities, but a strong QAS study still requires expertise in managing sample quality and controls that ensure quality inputs. Your insights depend on it!

Analysis and Reporting – Great Summaries, Beware of Information Overload

Most Qual-at-Scale platforms have robust analysis tools that can help quickly summarize interviews, including automated transcriptions, theme and sentiment analysis, video reels, and “chat with your data” capability. These tools lean into generative AI’s strengths: processing large quantities of data and giving good, if not great, summaries.

However, we found that, as with other LLM interfaces, it can sometimes draw broad, sweeping conclusions from small or nuanced findings. And because it’s focused only on the interview data, humans are required to provide the “so what” implications from the research. This further shows the importance in having an expert in qualitative research review the results.

Qual-at-Scale as a Bowl: Not Quite Quant, Not Quite Qual

Qual-at-Scale truly fits in a middle ground between the depth of traditional qualitative research and the breadth of quantitative. If qualitative is a water bottle (tall but narrow) and quant is a plate (wide but shallow), QAS is more of a bowl – medium width and depth, good for specific foods but not for drinks or full meals.

Let’s talk about how QAS compares to traditional methods:

Qual-at-Scale vs. Traditional Qual

Traditional qual interviews are still best for research that requires empathy and curiosity, as Qual-at-Scale is still fairly structured and rigid. Research that requires diving deep into a complicated topic, or one that involves technical language or terms, is better suited for traditional interviews. Sensitive topics that can elicit powerful emotions are also best handled with human empathy and judgment. Additionally, QAS is less helpful when the research is more exploratory in nature, such as ethnographies or foundational behavior research.

Qual-at-Scale vs. Quantitative

At its core, Qual-at-scale is still a qualitative exercise – that is, the data you get back from the research is words, themes, and sentiment, not numbers. Thus, QAS would not be appropriate for any research that requires statistical inferencing such as stat testing between subgroups, key drivers analyses, or any other higher-level modeling. QAS should not be seen as a substitute for quantitative research, even if it is performed at a quantitative scale.

Unique Solution for Unique Issues

Qual-at-Scale works best when researchers want to obtain moderate depth on a topic that is already mostly well-known or established. For example, QAS could work well to figure out how consumers feel when calling customer service, but less well on ideas for remaking the customer service experience entirely. Other situations QAS works well on include:

• Looking for hidden/emergent themes across a wider audience 

• Understanding language, metaphors, and framing that consumers use 

• Illuminating consumer segments and discovering micro-segments 

• Concept generation at early stages of innovation 

• In-depth review of final stages of concept development 

Our Verdict: Fantastic Tool in the Hands of Experts

Regardless of where it is used, however, it is important to remember that Qual-at-Scale is a methodology, not a cure-all. Like other research tools, it is still dependent on solid learning objectives based on what businesses need to make decisions. Without proper thought into research design, sample quality, and analytical frameworks, QAS can leave you very information-heavy but insights-poor.

As we continue to advise clients on where Qual-at-Scale fits best, we’re seeing its impact firsthand — supporting stronger, faster business decisions. If you’re exploring Qual-at-Scale and want to ensure it’s designed for high-quality learning, Burke can help you apply QAS in a way that strengthens decision-making, accelerates timelines, and keeps insight at the center.

Interested in reading more? Check out Jeremy’s other articles:

Helping Consumers Navigate Tariffs

Inflation’s Impact on Consumers

As always, you can follow Burke, Inc. on our LinkedIn, Twitter, Facebook and Instagram pages.

Source: Feature Image – ©Smile Studio AP – stock.adobe.com