CHOICES™
on the Web
Burke
Interactive programming and survey hosting makes it possible to use the
Internet for studies employing Burke’s CHOICES™
discrete-choice approach.
Case
History—Sprint’s Award-Winning Discrete Choice Research
Sprint
recently was named an honorable mention winner in the 2001 EXPLOR awards for an
online study employing CHOICES™.
The EXPLOR
awards are designed to salute “Exemplary Performance and Leadership in Online
Research” and are administered by the University of Wisconsin’s A.C. Nielsen
Center for Marketing Research and the American Marketing Association. Stan
Frear, group manager of Sprint’s consumer marketing research division, will
discuss the study at the 2001 EXPLOR Forum, to be held November 15-17, 2001, at
the Hotel Intercontinental in Chicago. Burke senior vice president Jeff Miller
will also present an overview of online research at the event.
Study
Overview
Technological
advances, regulatory changes and corporate mergers are bringing with them a
convergence of telecommunications and entertainment services. To help meet the
challenges of this convergence, Sprint worked with Breakthru Strategies, an
industry marketing consulting firm, to develop an overall approach to modeling
the rapidly evolving market.
A discrete
choice exercise using Burke’s proprietary CHOICES™ approach served as the
foundation for much of the model. A total of 1,500 members of a U.S. online
consumer panel were asked to select preferences from sets of individual and
“bundled” services offered by a variety of national and regional providers of
local, long-distance, and wireless telephone service, Internet access, and
cable television and other entertainment content.
The survey
was custom programmed by Burke Interactive so that service choices in each set
were randomized within a complex set of constraints, resulting in data on 9,000
unique choice sets. Using a disaggregate discrete choice procedure enabled
Sprint to predict what kinds of services individual consumers are interested in
purchasing – and what competitive conditions make it likely that they will.
Sprint has used the findings to make forecasts for specific communities with
different demographic profiles and can also adapt the model as national
demographic and technology-usage patterns change over the next several years.
More
Information
·
Study
Background: Telecommunications Convergence
·
Increasing
Respondent Understanding
Study Background: Telecommunications Convergence
Telecommunications
has seen a convergence of formerly disparate categories in recent years, with
single providers now frequently offering a “bundle” of services. Such a bundle
might include (in various combinations) local, long distance, and wireless telephone
service, conventional and high-speed Internet access, and cable television or
other entertainment content. Meeting the challenge of this convergence –
brought on by technological advances, regulatory changes, and corporate mergers
– is central to Sprint’s marketing efforts now and in the foreseeable future.
The landscape
has been evolving so rapidly, in fact, that it has been difficult to predict
what kinds of services consumers would be interested in purchasing and under
what conditions. Sprint has conducted highly sophisticated research on consumer
decision processes utilizing discrete choice modeling. This research has
considered the choices consumers make in each of the converging
telecommunications categories, as well as how these choices interact with each
other when multiple services are offered by the same supplier. Data, however,
were in some cases nearly obsolete before results were available and models
could be built. The competitive landscape represented no longer existed, with
new alternatives already available to consumers.
As a
consequence, Sprint undertook the present project to build a more flexible
model that could adapt to marketplace changes for at least five years into the
future. Moving beyond just marketing research, this effort involved creating an
entire framework for decision-making informed by multiple sources. This model
has proved to be effective for year 2000 forecasting, and shows promise to be
an adaptable and invaluable business tool for the future.
The
predictive model was populated by data from a discrete choice exercise
conducted by Burke Interactive among 1,500 members of a U.S. online consumer
panel. Based on models from other research, Sprint was specifically interested
in consumers falling in certain more technically-oriented segments of the
population, which by definition overlap largely with current users of online
services. Consequently, Internet-based data collection was very appropriate for
reaching the Web-enabled target market of interest. An Internet panel was
utilized as a sample source and a $10 incentive was paid.
Each
respondent viewed six different market scenarios (choice sets) in the discrete
choice exercise.
Unlike many
consumer choice models used in the telecommunications industry, this effort
recognized that even as more services are marketed more frequently as bundles,
many consumers will continue to opt to purchase them separately from different
suppliers. Possible product categories therefore included six “individual”
services (local telephone service, cable programming, Internet access, etc.),
as well as four specific combinations of these six services likely to be sold
as bundles (telephone service with Internet access, for instance).
Both of these
alternatives (bundled or separate purchases) had to be included to produce a
useful and comprehensive model of consumer behavior, even though doing so
introduced complications into the data collection process. Not all product
categories were available in each choice set, however, so that simulations
could be conducted to model “before” and “after” the emergence of new products
and estimate the impact of their introductions.
It was also
deemed likely that some of the products would be offered by more than one
supplier simultaneously, so within some choice sets, some products were
available in more than one version. Within each choice set, there were actually
between six and 13 product options.
The 10
product categories included some pairs which were mutually exclusive, such as
cable television alone and cable television bundled with telephone service, and
other pairs which were not mutually exclusive, such as residential telephone
service and dial-up Internet access. Consequently, while respondents viewed
themselves as choosing multiple services from a menu of up to 13 alternatives,
they were actually choosing one of 79 possible combinations of services (all
possible combinations with the mutually exclusive pairs not permitted).
Each of the
products was, in turn, defined by up to 11 attributes, including brand name,
both recurring and non-recurring costs, and features such as included minutes
of calling, upload and download speeds, and activation variables. Not all
attributes applied to all product categories.
One challenge
Burke Interactive faced in programming the study is that the largest providers
of many telecommunications services are local companies with local brand names.
To accurately capture the equity of these important local players, the brand
names appearing in interview stimuli were customized to each local market. The
interview covered 15 local markets, and each had its own unique providers (and
different numbers of providers) of local telephone service, cable television,
and Internet access.
Increasing
the design complexity further were several prohibitions in the combinations of
attribute levels that could appear in a given choice set. For instance,
although Internet access upload and download speeds covered overlapping ranges,
upload speed could be no faster than download speed. Certain speeds could only
be combined with certain technologies (DSL vs. cable modem, for instance), and
certain brands could only offer certain technologies (a cable company wouldn’t
offer DSL). The ability to share “included calling minutes” across platforms
(long distance with wireless, for instance) was contingent on both products
being available from the same supplier and the product offering actually
featuring some included minutes (yet another variable).
These
contingencies, along with the overall size of the design and model, made a
traditionally-constructed experimental design of a manageable size for the
discrete choice exercise virtually impossible to develop. Instead of using such
a static blocked design, one of the innovations of this research was employing
a fully-randomized design.
Each choice
set viewed by each respondent was created “on the fly” through random number
generation, according to a highly complex algorithm adhering to a set of rules
and constraints. With 1,500 respondents each viewing six choice sets, data were
collected on a total of 9,000 unique choice sets.
Near perfect
attribute level balance and orthogonality could be achieved where desired (each
level appearing the same number of times and paired equally often with the
levels of other attributes). Orthogonality was achieved not only between pairs
of attributes within a single choice option, but also between pairs of
attributes across different options. (In some cases, perfect balance was
not desired, and it was a simple matter to make certain levels for which more
data and precision was desired – “Sprint” as the level of “brand,” for instance
– appear more frequently.)
No aliasing
(confounding of main effects with high-order interactions between other
variables) was required, so all interactions, even those of higher order and
those not anticipated before data collection, could be estimated.
Increasing Respondent Understanding
Because of
the complexity of the respondent choice task (up to 13 potentially unfamiliar
options with up to 11 attributes each), several creative measures were taken to
increase respondent understanding and thus the quality of the data. Some of
these measures were enabled specifically by the online interviewing method.
Because some
of the products and features were unfamiliar, respondents were exposed to
full-color, annotated mock-ups of promotional materials – as realistic as
possible with brand logos where appropriate – before beginning the discrete
choice task.
Once in the
choice task, respondents viewed the choice sets one at a time. Each set was
presented as a grid, with names of various available options across the top and
the attributes describing each in rows beneath them. Appearing at the top of
the page were several “hyperlinks” which, when clicked, allowed the respondent
to return and review different aspects of the background promotional materials
in pop-up windows.
The
left-to-right order in which the specific options were presented was varied to
avoid any bias of order effects. This variation, however, was between
respondents rather than within the choice sets seen by a particular respondent.
For any one respondent, the order of the options was constant across the six
choice sets he or she completed to keep the task less confusing.
Because not
all options were available in all choice sets, a given option still did not
always appear in exactly the same location for each respondent. Since visual
space was at a premium and a goal was to minimize left-to-right scrolling by
respondents, when an option was not available in a given choice set, blank
columns were not left in the grid. Rather, as some options moved in or out of
the set, others were naturally shifted to the left or right with a variable
number of columns displayed.
To help
respondents locate familiar products from set to set, color coding was used.
The attribute text in the column describing each option was presented in a
unique color that remained constant across choice sets. The color coding scheme
was carried forward to the bottom of the page, where respondents actually
clicked to select the services they would purchase, to help ensure that they
were selecting the intended alternatives.
For the
analysis, a 79-option discrete choice model was estimated (one option for each
of the 79 possible combinations of services). Burke Interactive provided a
stand-alone simulator based on the research sample. Because Burke’s proprietary
CHOICES™ discrete-choice modeling was used to analyze consumer preferences at
an individual level, choice tendencies could be presented as a function of a
variety of demographic and current technology-usage variables.
The discrete
choice exercise also served as the foundation for most of the parameters and
effects in the larger flexible structural model developed by Breakthru
Strategies. That framework also incorporates other estimates, especially of the
impact of products and services not currently available. These products may be
too foreign to consumers for them to provide reliable data about their behavior
with regard to those options. As these products gain greater penetration,
additional research will be used to refine the existing model, so that there
will be no need to start over from scratch.
Of course,
even among currently familiar products, prices and usage patterns will change,
and consumers will over time assign different priorities to various options.
The disaggregate nature of the model derived from the consumer discrete choice
exercise allows key parameters to be readily updated. Because, for example, the
relationship between demographics and preferences has been drawn, demographics
themselves can be directly inputted into the model. The model can therefore
adapt as national demographic and technology-usage patterns change over the
next several years, and it can also be customized for specific communities with
different demographic profiles. Such flexibility, for example, helped Sprint
perform short-term market-specific planning in the fall of 2000.