What is MaxDiff?

Facilitating effective decision making using MaxDiff scaling

The road to bringing a new medical device to market is full of tough questions and difficult decisions. What are the most significant customer needs in todays’ market? What products, features, or services should be prioritised for development first? How can we market this new product most effectively, in order to maximise sales?  It can be all too easy to end up drowning in a sea of insights, with plenty of exciting ideas, yet unsure which will deliver the greatest return on investment. For example:

• Unmet needs uncovered in early stage qualitative/ ethnographic research
• Potential features of a new product or service
• Potential benefits of a wide range of new potential products or services
• Draft marketing messages

MaxDiff involves presenting respondents with a series of best/worst prioritisation tasks, each task containing a subset of items from the longer list (usually 4-5). Respondents are simply asked to select the best and worst items from each set. An example task is shown below:Imagine your Doctor recommended a change in your current diabetes treatment.
We would like to understand what you would need in order to help you to adjust to the new treatment and to make the ongoing experience of managing your diabetes as positive and straightforward as possible.
On the following screens, you will see a selection of needs that other people with your condition have said are important to them when moving to a new diabetes treatment. For each set of needs shown, please select the ONE that is MOST and LEAST important to you.

In subsequent tasks, the way in which different list items are grouped together is controlled using an algorithm. The data is then analysed using a method known as Hierarchical Bayes estimation, to provide importance scores for all of the items relative to one another, even though all items are never shown together. The scores are then indexed to 100 to aid interpretation. The result is a clear prioritisation of market preferences as can be seen in the figure on the right.MaxDiff has several key advantages when compared to other more traditional commonly used methods for item prioritisation, namely ratings scales and ranking approaches.

Ratings scales
With ratings scales, respondents are asked to score each item on a scale of importance, which for example might range from 0 (not at all important) through to 7 (extremely important). This is simple enough in theory, but in multi-market studies often proves problematic as there are distinct cultural biases in how such scales are used.For example, compared to European respondents, Chinese and Japanese respondents tend to utilise the top end of the scale much more, leading to apparent differences in strength of preference between markets that are in fact meaningless to interpret. With MaxDiff, respondents are forced to make trade-offs rather than indicate strength of preference, so this issue is completely avoided. Additionally, ratings scales frequently fail to discriminate effectively between items, with results often showing many items as being ‘highly important’.This may well be valid, however it gives little direction for product developers when budgets are capped and only a small subset of ideas can be taken forward. The key issue here is that because each item is considered in isolation, respondents are not forced to prioritise. Again, MaxDiff overcomes this problem by forcing respondents to consider what is most and least important, leading to better discrimination between items in the final results.

Finally, it can be argued that MaxDiff results are likely to be more reflective of real-life decision making behaviour than ratings. In real life, we make choices based on the information available to us – we don’t sit there and score each marketing message we see based on how much we like it.

MaxDiff and scale bias

Of course, asking respondents to rank their preferred items in order of priority both avoids the issue of scale bias and forces respondents to prioritise. However, this approach has its own pitfalls:

Firstly, because all items must be presented and evaluated simultaneously, only a relatively small number of items can be assessed. Each extra item included increases the difficulty of the task, and this can easily lead to lack of respondent engagement and poor data quality.

MaxDiff allows a much larger number of items to be considered as only 4-5 items are required to be evaluated at once. What is more, the choice tasks are also simpler for respondents to complete – rather than needing to prioritise all the items in a task, they are only required to select ‘best’ and ‘worst’. Second, rankings provide purely ordinal data, that is, we know 1st is better than 2nd, but not how much better – there could be little difference between them, or, 1st could be way out in front. MaxDiff provides interval data, allowing us to clearly differentiate the ‘top tier’ priorities from those of lesser importance.


MaxDiff – limitations

All this said, MaxDiff itself has limitations. While it will deliver a clear hierarchy of relative priority for your tested items, MaxDiff alone will not tell you in an absolute sense how important a particular need is, or what percentage of people like your potential product feature; the list item with the highest MaxDiff score could either be the next big market game-changer, or just the best of a bad bunch of ideas.

Thinking of using MaxDiff in your next research project?
Therefore, as always, it’s important to ensure your research objectives are clearly defined at the outset. If required, MaxDiff scores can easily be augmented with other data in order to provide such insights. If you’re considering a study, talk to us and we’ll guide you through the best ways to maximise your study using MaxDiff. It’s not for everyone however, but you’ll see the benefits to a study when used in the right settings.