Using Conjoint Analysis to Answer Strategic Questions: Pricing Events

During our Pricing and Profit Management Winter ’24 Tour, we asked attendees to fill out a marketing survey. I reviewed the initial survey results in last month’s Wiglaf Journal. This month, I get to talk about the analysis that I completed using another marketing analytical tool that can really provide deep insight into customer preferences: conjoint analysis.

What is conjoint analysis?

Conjoint analysis arises from the concept of conjoint measurement, which was a method proposed by mathematical psychologists in the 1970s. Basically, it posits that buyers:

evaluate the overall desirability of a complex product or service based on a function of the value of its separate (yet conjoined) parts… By systematically varying the features of the product and observing how respondents react to the resulting product profiles, one can statistically deduce… the scores (part-worths) for the separate features respondents may have been subconsciously using to evaluate products. (Orme, p. 1–3)

Respondents select a given offering because they determine that it has certain levels of specific attributes that are more valuable or more appealing to the respondent than those of other offerings. And every offering can be broken down into a stack of specific attributes or features that distinguishes it from other offerings. And respondents will choose the offering stack that best meets their needs.

Traditional conjoint analysis involved rating or ranking all offering profiles. In the 1990s, choice-based conjoint (CBC) emerged on the scene. CBC shows respondents a pair of offering profiles at a time, and the respondent must select between the two. Some surveys also allow selecting neither profile as an option.

We used CBC for our marketing survey.

Why use conjoint analysis?—high-level managerial view

First, choice-based conjoint (CBC) analysis requires respondents to select between pairs of products. This means that the respondent is always comparing each offering to an alternative, forcing them to make tradeoffs in their decisions. This mirrors how people make purchasing decisions in real life.

Second, CBC analysis dramatically cuts down on the number of prompts that a respondent must select between. Instead of reviewing hundreds of possible offering profiles, CBC only requires the respondent to view a dozen or so. (More details in the data nerds section below.)

Finally, using conjoint analysis allows you to construct various market simulations, from which you can calculate shares of preference for various offering stacks. Want to see what happens if there are 3 alternatives on the market and a 4^th shows up? Want to see where customers are likely to shift if one of the 3 alternatives disappears from the market? Want to get an idea of which features your R&D and product management departments should be focused on based on customer preferences? A market simulator can help with that.

Strategic questions answered

Goal:

Determine the optimal mix of event attributes that will result in the highest number of event attendees.

Method:

Conjoint Analysis

Attributes Tested:

Time: 9 am, 12 pm, 3 pm, or 6 pm

Food: hors d’oeuvres or meal

Price: $15, $25, or $55

Details:

34 total respondents, tested using pencil-and-paper surveys in Chicago, Minneapolis, Kansas City, and Bentonville

Result:

We have decided to change the format of our events this summer. We will host events at 3 pm local time with hors d’oeuvres and beverages at $15 for the Early Bird, increasing to $30 in the final 3 weeks before the events.

Specific results

In coming to our decision, we relied upon two important bits of the analysis: the relative utilities of the individual attributes and market simulations of the most popular options. The former information is important, but I think the more insightful portion is the latter.

This is because the value of each individual attribute is interesting information, but the most important thing is how all the individual attributes combine into an overall offering that the respondent evaluates. And specifically, how does the respondent consider the offering versus the alternatives?

To give you a taste of the former, here are the relative shares of preference for the two meal options that we tested:

This shows that our respondents preferred hors d’oeuvres to a meal by a 2:1 ratio. That is an important insight!

So, it appears as though we should alter our offering to have only hors d’oeuvres instead of a full plated meal. But what about the other factors? What time of day should we have our event and at what price point? And how do these other factors interact with the meal factor?

The really powerful insights are found in the market simulations. Because we were previously doing our events at noon with a meal for $55, we tested that option vs. 2 alternative times for hors d’oeuvres for $15. We wanted to see what the marginal improvement would be for each time. Below are the results of these 2 scenarios.

First, would our attendees prefer a meal at noon for $55 or hors d’oeuvres at 6 pm for $15?

As you can see, our attendees came down 57:43 in favor of the hors d’oeuvres at 6 pm. These results indicate that we could improve our attendance by 33% simply by scheduling the event 6 hours later, downgrading the food, and reducing the price. But what about trying the event at 3 pm instead?

As the chart above shows, our attendees would prefer hors d’oeuvres at 3 pm over our previous event format by a 64:36 ratio. This means that we could potentially improve our attendance by 78% by making the event only 3 hours later, downgrading the food, and reducing the price. That’s a pretty strong argument for change.

Thanks to the insights from conjoint analysis, we have our event format settled for our Pricing and Profit Management Summer ’24 Tour.

Why use conjoint analysis?—in the weeds, for the data nerds

The technically amazing thing about conjoint analysis is that it offers robust measurement of shares of preference while also dramatically decreasing the number of offering concept pairs that a respondent must choose between.

How does it do that? Through the magic of orthogonal fractional-factorial experimental designs, of course! But that’s quite a mouthful. What does that even mean? Well, because these designs are orthogonal, they allow you to effectively eliminate a dimension of the analysis without sacrificing the statistical rigor of the results.

Imagine taking a 3-dimensional cube that you need to analyze, eliminating the z-dimension, and shrinking it down to a 2-dimensional square. Or imagine taking that 2-dimensional square, eliminating the y-dimension, and shrinking it down to a 1-dimensional line. In each case, you dramatically decrease the possible analysis space because you are removing a dimension from the analysis.

Likewise, these orthogonal designs allow us to smartly and efficiently tease out shares of preference with a much smaller survey, i.e., by presenting far fewer offering concepts pairs to respondents.

In order to appreciate the advantage of a fractional-factorial design, consider the full-factorial design. A full-factorial experimental design would involve creating an offering prompt for every single possible combination of attributes. Then, you test each prompt against each other. Basically, you test every possible pair. For our combination of attributes, that would mean a total of 24 offering concepts to test:

And if you want a respondent to choose between each and every pair (i.e., take each of the 24 concepts and methodically pair it with the other 23), then there are 276 possible 2-pair combinations:

Note that we divide by 2 because the order of the concepts does not matter in the pair. E.g., if you have the letters A, B, and C and you choose 2 letters at a time, then you have 3 x 2 = 6 permutations: AxB, AxC, BxA, BxC, CxA, and CxB. But, if order doesn’t matter, then AxB = BxA, and AxC = CxA, and BxC = CxB. Thus, you only have half the possible pair combinations as you have permutations.

There is no point in showing concept #1 vs. concept #2 and then concept #2 vs. concept #1. The order of the concepts should not affect decision-making; thus, order does not matter.

So, a full-factorial experiment design would require presenting each respondent with all 276 possible pairs so they can choose between them. I hate to break it to you, but the only thing this survey will accomplish is curing your respondents of insomnia.

But how many 2-pair combinations did this conjoint analysis require? Drumroll please… This conjoint analysis required: 12 pairs of offering concepts.

That’s right. Conjoint’s orthogonal fractional-factorial experiment design shrunk the number of paired offering concepts that we presented to our respondents from 276 to 12. And having respondents choose from these 12 pairs provides a statistical analysis that’s as accurate for our purposes as if the respondents had scored all 276 possible pairs.

That’s the power of conjoint analysis.

(For a more detailed description of fractional-factorial design, including an example of coding a traditional conjoint analysis into Microsoft Excel and analyzing it using dummy variables and Excel’s multiple regression tool, see chapter 8 of Bryan K. Orme’s Getting Started with Conjoint Analysis: Strategies for Product Design and Pricing Research.)

References

Orme, Bryan K. Getting Started with Conjoint Analysis: Strategies for Product Design and Pricing Research. Manhattan Beach, CA: Research Publishers LLC, 2014.