MaxDiff

MaxDiff, also called best-worst scaling, ranks a set of items against each other on a single dimension. Instead of rating every item on a scale, respondents see a small subset of the list and pick the most important and the least important. The subsets rotate so respondents compare each item in different combinations. The output is a set of preference scores you can act on.

When to use MaxDiff

• You have roughly 15 to 40 items and need to know their relative order
• The items are comparable, meaning they sit at a similar level of abstraction. All features, or all claims, not a mix of both.
• The decision is about priority: what goes first, what gets cut, and what leads the message

If you need absolute levels rather than relative order, use a rating scale. If the list is 5 items or fewer, use a Ranking question. If you are testing combinations with attributes and prices, use conjoint.

Adding a MaxDiff question

1. Go to your survey's Build view.
2. Choose Add question, then More, then MaxDiff.
3. Enter your question text. The default is "From this set, select the least important and most important option."
4. Add your items. Each item is a row in the list. Keep items short and parallel in structure, and at the same level of abstraction.

Settings

Items per set is how many items a respondent sees at once. 3 to 5 is typical. The default is 3. Fewer gives you less information per choice. More increases the reading load without improving the estimates.

Sets per respondent is how many sets each respondent completes. The default is 4. Respondents typically complete 8 to 15 sets, so raise it until coverage lands in range.

Worst label and Best label set the column headings respondents see. The defaults are "Least important" and "Most important." Change them to match your question, for example "Least appealing" and "Most appealing."

Labels controls where item text sits between the two columns.

MaxDiff questions also support Optional, Subtext, Minimum time, media, images on choices, Display logic, and Skip logic, the same as other question types.

Coverage

The most common way a MaxDiff fails is a design where items do not appear often enough for the scores to be reliable. 

As you change Items per set or Sets per respondent, the coverage panel shows the average number of times each item will appear per respondent and the arithmetic behind it: items per set multiplied by sets per respondent, divided by the total number of items. Aim for each item to appear 3 to 5 times. Below 3, the scores get noisy. Above 5, you are adding length without adding much precision. The panel tells you whether your design is in the recommended coverage range before you field the survey.

Results

Results come back as preference scores that sum to 100 across the list, so the distances between items are readable. An item scoring 14 is worth roughly twice as much to your audience as an item scoring 7. Scores are ready to read in Analyze without a separate analysis step.

Availability

You can build out a MaxDiff question in any survey. However, you'll need to purchase the Advanced Methods add-on to unlock the ability to collect responses.