Product research is a general term that encompasses a wide array of custom designs and applications, with the common thread being the evaluation of a product (or service) by the end-user or target customer. Following is an outline of Burke's view of product research, including some of the more common methodologies used by Burke in the product testing arena, presented in order to understand their respective strengths and limitations.

Burke has great depth of product research and testing experience, spanning a wide range of both consumer packaged goods and non-packaged goods, such as PC software, new types of financial accounts offered by banks or brokerage firms, or a new phone system for a business. The utilization of the discussed test designs that follows will use a consumer food product as an example (e.g., ice cream sandwiches) but would still apply to other consumer or business-to-business services and durables, in most cases, with just slight modifications.

It is initially important to realize that the ultimate decision on which research design to use will, of course, depend upon the specific objectives of that particular test. No single design can unequivocally be considered "the best". In most cases, the objectives of the test will be contingent upon the development or life-cycle stage of the product being tested.

In the early stages of product development, only a prototype may exist, and the objectives typically focus on the optimization of features and characteristics that would maximize customer appeal. In addition, product research can be used to identify the positioning strategy that best translates the product features into salient consumer benefits.

Let's assume our hypothetical ice cream sandwich marketer has determined via concept screening that consumers have an interest in an ice cream sandwich made with mint chocolate chip ice cream, instead of vanilla. An initial product test should determine if the taste meets the expectations. In essence, the goal is to confirm that what sounds like a good idea can be properly executed. Data from a product test would also allow fine-tuning of the product attributes to insure the brand had the right texture, creaminess, sweetness, strength of flavor, etc

When a final or finished product exists, but still prior to introduction, a test of the product versus the competition is frequently beneficial. This serves to identify the competitive strengths (hopefully many) and weaknesses (hopefully few), as well as confirm that the brand positioning is on target. Identification of the attributes that drive preference of your brand over the competition can then be leveraged in advertising.

Once on the market, product testing of established brands is usually conducted with one of two purposes in mind. First, as a quality control measure, with the goal being to maintain the standards of the delivered features over the life of the brand. Second, if potential improvements can be made to the product, a reformulation is tested.

The area of product reformulation testing is quite common and can also be divided into two general areas. First, the product may be reformed in order to capture additional market share, as evidenced by promotion of "new and improved" features. Here, the objective is to determine if the reform is truly superior to the original.

The second area is cost reduction reformulations. Typically, the product manufacturing process is changed, perhaps through improved technology at the factory, or the substitution of a less expensive raw material or ingredient. Assume the brand manager of our hypothetical ice cream sandwich has found a lower cost supplier of chocolate chips. The end result is a product which is technically "different" from the original, but hopefully exactly the same from the customer standpoint. While switching suppliers would result in significant savings to the bottom line, it is desirable to insure the higher profit margin is not offset by a loss of share due to product dissatisfaction. Indeed, reformulations of this type are introduced with no promotion, and the objective of a product test is to determine if consumers can discriminate between the original and the reform. Testing among the franchise of your brand users is necessary to meet this objective.

While the above situations call for product testing to answer a marketing issue, there is still the design of the product test itself to be decided. There are two basic product test designs that are commonly used in research, the monadic and the paired comparison.

Monadic Product Tests. In monadic testing, a respondent tests a single product and provides an evaluation of that product. Data collected typically includes variables such as purchase interest and ratings on attributes. If there is more than one product to be tested, matched groups of respondents would test each product, with the data collected from each group being compared to each other.

Paired Comparison Product Tests. In a paired comparison test, respondents use two products in sequence, with no questioning in between. After both products have been used, they are asked to rate each and state a preference. Because questions are not asked until both products have been tried, the evaluation of a product is always in the context of a comparison to the other.

There are other comparative type test designs, such as triangle and triad tests (evaluating three products in sequence), as well as variations such as repeat paired comparison designs, but they are all essentially modifications of comparative testing designs. Generally, however, when deciding upon the appropriate test design, the first decision must be "monadic" or "comparative"?

Choosing The Right Research Design. As usual, the answer depends upon the objectives. In the prototype stage, monadic testing is usually the best option, since the objectives often require obtaining more basic data on overall product appeal (i.e. do people like it or hate it?). The monadic test is also preferable when a direct competitor does not exist or is difficult to identify. When measuring product performance over time the monadic test is superior. The reason for this is because the paired comparison test only provides information relative to the alternative products being tested. The monadic test, however, provides information relative only to the respondent's internal judgment. Therefore, this information can be compared to other monadic information obtained in the future (assuming comparable samples). Finally, the monadic test is also regarded as more "real world", since it replicates the fact that consumers generally use one product at a time.

The comparative test is recommended when the objectives call for declaring a "winner" (for example, choosing between several prototype variations of the same product). Because more than one product is being tested by a single group, differences between products tend to be magnified, and are therefore easier to detect. In some instances this result is desirable, but it can also be potentially dangerous when the differences are heightened out of proportion to their importance in the market. Obviously, when information is needed with respect to the competition, a comparative test is superior. Testing advertising claims (such as "preferred to the leading brand") are almost always comparative designs.

Two factors which always affect the ultimate research design are cost and timing. In budgetary terms, the comparative method is more economical. Not only do monadic tests require matched independent samples for each product, but a greater number of interviews is needed to detect differences between monadic samples versus the single group sample of a comparative test.

Though less expensive, comparative tests take longer to complete. In some instances, such as a central location taste test, evaluating two products takes only marginally longer than evaluating one, with the difference measurable in minutes. For an "in-home" or "in-office" use test, however, the incremental time needed for the paired comparison can be measured in terms of days or weeks.

Regardless of the design, several additional decisions must be made when outlining the test methodology. One question is should the test product or products be placed with respondents to be used at home or in the workplace or other relevant location, or should the product(s) be tested at a central location (often abbreviated as "CLT" for "central location test")? Another is should the product(s) be tested blind, meaning without respondent knowledge of the brand name, or should the brand name be obvious? The reader can probably by now predict the answer to each…..it depends on the test objective.

The in-home or in-office/workplace usage is certainly more realistic, and provides greater face validity. But the CLT allows for much superior control of the testing process, which hopefully results in a reduction of non-sampling error.

Blind or unbranded tests will allow for measurement of the true physical performance of the product, unencumbered by the equity associated with brand name. A familiar and well regarded brand name can produce a favorable "halo effect" on the evaluation of the product. In a sense, product performance can become more a function of the brand image, than the product itself. For this reason, claims tests of product superiority over the competition are almost always blind. However, branded testing is more realistic, and offers increased predictive validity vis-a-vis a blind test.

Because both the monadic and paired comparison designs have a number of strengths, a hybrid design alternative called the sequential monadic was developed to provide both the "absolute judgment" data of a monadic and "comparative" type data of paired comparisons. In a typical sequential monadic test, a respondent evaluates two products in sequence. The first product is evaluated without knowledge a second product will follow. After rating the first product, the second product is evaluated. The questions asked about each product are usually identical. At the very end, a preference between the two products is obtained.

Burke has identified that the "typical" way of executing this type of test design has a major flaw. Once the first product has been evaluated, the ratings of the second product are in reality no longer truly monadic. The respondent has become familiar with the questioning sequence as a result of the first product evaluation. More importantly, while the ratings of the first product reflect the consumers absolute judgment, with no other product as a basis for comparison, it is impossible to evaluate the second product in this manner. Unquestionably, the consumer will evaluate the second product in relation to the first, even though comparative questions are not asked until later.

While rotation of the product tested first and second can equalize this bias, eliminating it entirely is preferable. Therefore, Burke improves the design by omitting the second monadic evaluation. The end result is a design that obtains true monadic data on each product, as well as comparative data on preference. While widely known as "sequential monadic", a more accurate description would be "monadic with comparative tag-on".

An advantage of this design is that monadic data are frequently enhanced by having comparative data, and vice versa. For instance, if monadic data indicate "no differences" between our two prototype ice cream sandwiches, the comparative preference data can serve as a tie-breaker for decision making purposes. Conversely, if only comparative data existed between the two prototypes, we would know which product is preferred, but not whether either is acceptable. In effect, with only comparative data existing, we could be finding what is the best among bad alternatives.

In conclusion, product testing is an essential component of successful marketing. Reliable customer data about a "product" maximizes the probability of market success, while minimizing the probability of a market mishap. Monadic designs, comparative designs, and hybrids of the two all have their place in the product testing arena. A clear understanding of the objectives should insure selection of the proper technique.

When direct preference is of primary interest in a product testing situation, Burke has extensive experience applying the Repeat Pair method. This method is often recommended by Burke because it provides the most reliable measurement of taste testing preference. Such a design simultaneously provides a measure of preference for each product in a test pairing, as well as a clear estimate of the number of true "non-discriminators" who exist in the target market. The "non-discriminators" are those persons who truly cannot tell the difference between the product alternatives, or those whose preferences are so weak they will vacillate between the two products. Isolating non-discriminators allows focus on the responses of those who consistently prefer one product over the other. Burke's Repeat Pair technique focuses on both direct product preference and on product diagnostics. The net result is maximum ability to answer the questions: "which product is preferred?", and "why?"

A basic example may help clarify the need for isolating non-discriminators in preference testing. In a single pairing of products, 250 consumers taste Product A and B (rotated). When asked which product is preferred, 125 indicate Product A and 125 indicate Product B. From this pattern of results, two equally plausible conclusions are possible:

• The two products are equally preferred and the market is strictly segmented on the basis of preference.
• The two products are indistinguishable and preference is a random choice on the part of consumers; the market is completely homogenous.

A second pairing of the products (that is, having the respondents test the same pair again, without knowing that it is the identical set of products as before) produces four possible outcomes:

• Those who consistently prefer Product A
• Those who consistently prefer Product B
• Those who are inconsistent and switch preference from one tasting to the next
• Those who consistently express No Preference

The Burke Repeat Pair analysis provides estimates for each of the outcomes above, and the degree to which observed differences are both statistically and managerially significant. Depending on the specific research issue, the levels of discrimination and non-discrimination can have a substantial impact on business decisions. For example, if the research objective is to determine the preference between a reformulation and an original or previous version of a product, a high level of non-discrimination would be desirable, since the reformulated product could successfully replace the older product. Alternatively, a high level of preference for the original version may mean "back to the drawing board." On the other hand, in a test versus a competitor's product, a high level of discrimination, and thus preference, in favor of a client's product is most desired, indicating that a distinct competitive advantage has been achieved. Consequently, management may consider increasing marketing expenditures to aggressively promote this advantage. Using the Burke Repeat Pair technique greatly improves the marketer's ability to understand the true nature of product preference and thus make better business decisions.

Experience and Database. Burke has conducted over 250 Repeat Pair product tests, and maintains a database of results across a wide range of consumer packaged goods.

A Fortune 100 manufacturer of food products identified two cost reduction alternatives for one of their major brands. One new formula (call it "A") represented a 50% reduction of a key ingredient. The other new formula (we'll call it "B") also consisted of a 50% reduction in the ingredient, but also included slightly increased amounts of several other ingredients. While both new versions represented significant savings in the total manufacturing costs, the first alternative offered the greatest savings. In order to determine if either new product could replace the current, a Repeat Pair Taste Test was conducted to determine if consumers could notice any difference in the taste of either product versus the current formula. The test results indicated a high degree of non-discrimination (over 80%) between the current and alternative "A". In addition, among the minority of discriminating consumers, the new product was actually preferred. Management could feel comfortable marketing the brand under the new formula without negative consequence. However, new formula "B" represented an even better option. While many consumers (65%) still could not discriminate between the current brand and "B", the remaining discriminating consumers all preferred "B". Not only did "B" reduce manufacturing costs, but it was a truly superior tasting product as well, and could replace the current formula under a "new and improved" flag.

Consumer acceptance testing is a critical step in the development of successful products. Frequently, however, such testing does not provide adequate guidance for improving products and maximizing potential buyer appeal. Traditional analytical methods associated with product research (regardless of overall research design, i.e., monadic, paired comparison, sequential monadic, etc.) can suggest too many product changes, and fail to identify the changes that will be most critical to successful product improvement. Also, traditional analysis methods tend to identify only the direction of change, and not the degree of change that is needed to improve consumer acceptance.

JURI is a Burke proprietary analytical tool that directly relates the impact of each product characteristic to consumer purchase intent or preference. It is especially designed for use with the most common type of diagnostic ratings found in product testing, the "just right scale". A "just right" scale is an interview question that asks a respondent if a product is too extreme or "just right" with respect to an attribute. For example, a test participant might be asked if a food product is too sweet, not sweet enough, or just right.

Applying appropriate mathematical modeling techniques, JURI unlocks the diagnostic and directional power of product test results that is often hidden by traditional, simplistic analytical methods.

Specifically, JURI delivers:

• Specific recommendations for which product characteristics warrant improvement and which do not
• Clear priorities for improvement
• Concise and straightforward presentation of results
• A standardized approach, providing comparability across tests