Facilitating effective decision making using MaxDiff scaling
The road to bringing a new medical device to market is full of tough questions and difficult decisions. What are the most significant customer needs in todays’ market? What products, features, or services should be prioritised for development first? How can we market this new product most effectively, in order to maximise sales? It can be all too easy to end up drowning in a sea of insights, with plenty of exciting ideas, yet unsure which will deliver the greatest return on investment. For example:
With ratings scales, respondents are asked to score each item on a scale of importance, which for example might range from 0 (not at all important) through to 7 (extremely important). This is simple enough in theory, but in multi-market studies often proves problematic as there are distinct cultural biases in how such scales are used.For example, compared to European respondents, Chinese and Japanese respondents tend to utilise the top end of the scale much more, leading to apparent differences in strength of preference between markets that are in fact meaningless to interpret. With MaxDiff, respondents are forced to make trade-offs rather than indicate strength of preference, so this issue is completely avoided. Additionally, ratings scales frequently fail to discriminate effectively between items, with results often showing many items as being ‘highly important’.This may well be valid, however it gives little direction for product developers when budgets are capped and only a small subset of ideas can be taken forward. The key issue here is that because each item is considered in isolation, respondents are not forced to prioritise. Again, MaxDiff overcomes this problem by forcing respondents to consider what is most and least important, leading to better discrimination between items in the final results.
Finally, it can be argued that MaxDiff results are likely to be more reflective of real-life decision making behaviour than ratings. In real life, we make choices based on the information available to us – we don’t sit there and score each marketing message we see based on how much we like it.
MaxDiff and scale bias
Of course, asking respondents to rank their preferred items in order of priority both avoids the issue of scale bias and forces respondents to prioritise. However, this approach has its own pitfalls:
Firstly, because all items must be presented and evaluated simultaneously, only a relatively small number of items can be assessed. Each extra item included increases the difficulty of the task, and this can easily lead to lack of respondent engagement and poor data quality.
MaxDiff allows a much larger number of items to be considered as only 4-5 items are required to be evaluated at once. What is more, the choice tasks are also simpler for respondents to complete – rather than needing to prioritise all the items in a task, they are only required to select ‘best’ and ‘worst’. Second, rankings provide purely ordinal data, that is, we know 1st is better than 2nd, but not how much better – there could be little difference between them, or, 1st could be way out in front. MaxDiff provides interval data, allowing us to clearly differentiate the ‘top tier’ priorities from those of lesser importance.
MaxDiff – limitations
All this said, MaxDiff itself has limitations. While it will deliver a clear hierarchy of relative priority for your tested items, MaxDiff alone will not tell you in an absolute sense how important a particular need is, or what percentage of people like your potential product feature; the list item with the highest MaxDiff score could either be the next big market game-changer, or just the best of a bad bunch of ideas.
Thinking of using MaxDiff in your next research project?
Therefore, as always, it’s important to ensure your research objectives are clearly defined at the outset. If required, MaxDiff scores can easily be augmented with other data in order to provide such insights. If you’re considering a study, talk to us and we’ll guide you through the best ways to maximise your study using MaxDiff. It’s not for everyone however, but you’ll see the benefits to a study when used in the right settings.
20th September 2016