Quantiment®: The Best Friend to an Open-End

by Chris Deinlein

As researchers, we would love to engage with customers and prospective customers, asking questions about every aspect of their experience. However, we can’t keep them in a survey forever. We need to leave them some time for other of life’s priorities, although I think we can all agree that marketing research should be life’s #1 priority!

Due to the limited amount of time we have with customers and prospective customers, the open-ended question has found a special place in the hearts of researchers. By asking an open-ended question, we allow the customer to tell us, in his or her own words, how they feel about their experience. However, open-ended questions don’t quickly, or easily, translate into simple key drivers or metrics we can use to make business decisions. Quantifying an impact on NPS, Satisfaction, Re-Purchase, or any outcome measure is much easier with agreement scales or other familiar tactics.

When we analyze coded open-ended themes, we can evaluate the impact these themes have on our outcome measure of choice…rather than simply relying on frequency.

Through manual coding, or complex coding algorithms that take inordinate amounts of energy to create, we can group open-ended comments into themes to gain some degree of consistency across comments and examine results in aggregate. This coding process allows us to then analyze the data by looking at which themes are most frequently mentioned.

However, what if the items mentioned most frequently represent a “table-stakes” theme? These are the topics that receive a lot of airtime, but don’t actually impact the customer’s overall experience. What if we generally satisfy our customers, but then, when they mention having to email customer service 10+ times to resolve an issue, it has a major impact on retention rate.

These considerations throw a wrench into the traditional belief that higher frequency comment themes should be prioritized over lower frequency themes.

Recently, we developed Quantiment® (Quantitative Analysis of Customer Comments) to help solve for this challenge. Now, when we analyze coded open-ended themes, we can evaluate the impact these themes have on our outcome measure of choice – whether it be NPS, Satisfaction, Re-Purchase, or something else – rather than simply relying on frequency. In some instances, we already have an existing key driver analysis that is based on great attribute ratings and the associated impact each of these has on an outcome measure. However, what we might find is that it’s hard to add context on what needs improvement within each of those attributes to drive scores upward. Quantiment provides this additional context, as it examines the relationships between the open-ended themes and each attribute helping to quantify the impact of each theme to improve each attribute. This allows us to be more tactical in how we try to affect change in key drivers, while also incorporating an additional layer of the customer’s voice to bring the story to life.

In addition, Quantiment provides a more useful avenue for optimization. Let’s think of this in the context of a traditional key driver model. The first step provides a prioritization list that is based on the rank order of impact. The attributes or themes with a higher associated impact on our outcome measure would be prioritized according to that rank order. See the case study example below.

But what if the top three or so drivers are all related to one another, and improving one would result in a corresponding improvement in the others? Quantiment allows us to account for this overlap in themes, or drivers, to create an optimal path that gives more “bang for your buck” than just running down the list of most impactful drivers.

Think about this scenario – What if, instead of providing your stakeholders with a list of the most frequently mentioned themes, you can provide them with a strategy on how to most move the needle on your outcome measure by focusing on the words of your customers? Well, Quantiment was built for just that!

Open-ended questions really do provide great insights into how the customer feels about their experiences in their own words, but with Quantiment, we now have a streamlined way to convert those insights into strategies.

So, what’s the bottom line? Let those open ended questions fly!

As a Senior Analyst, Chris Deinlein offers his clients a fresh perspective on their data. Packaged in a storytelling bundle, Chris leverages various analytical techniques to bring insights to light.

As always, you can follow Burke, Inc. on our LinkedIn, Twitter, Facebook and Instagram pages.

Sources: Feature image (©PeopleImages –


500 WEST 7TH STREET | CINCINNATI, OH 45203 | 800.688.2674