Comments on Card sorting: a definitive guide

Donna Maurer — Fri, 26 Jan 2007 14:51:55 GMT

Thanks Anna – good question

In sorts that I have done where both groups and individuals participated, the only difference between the results is that those done by groups are a little more internally consistent (because there has been discussion about placement), and I obtain richer information about the reasoning for the groups. Overall the resulting groupings aren’t wildly different.

The more important issue is that individuals use the web *to undertake tasks* not to group content. This doesn’t necessarily bias the results (and it’s not a scientific study, so perhaps bias is irrelevant anyway), but it can mean that the results from the sort don’t allow easy task performance. That’s why we emphasise that it isn’t a silver bullet, and you get just as much (or more) insight from other user research activities.

For your second question, I don’t have examples as most of my work is on intranets, and examples wouldn’t make any sense out of context. Maybe Todd will have some, or another reader ;)

Anna — Fri, 26 Jan 2007 14:51:54 GMT

Two other questions as I review the article:

First, do you have any sample lists that you used for card content? That is, can we see examples of what would go on the cards for a particular site?

Anna — Fri, 26 Jan 2007 14:51:54 GMT

Great article—both informative and thought-provoking!

You recommend group card-sorting (groups of three) over individual sorting for efficiency and richer data. But what about the idea that _individuals_, not groups, use the web? To what extent does group think bias the results?

Thanks again!

Anna — Tue, 06 Feb 2007 11:46:11 GMT

Great article—both informative and thought-provoking!

Thanks again!

Marsha Waren — Fri, 26 Jan 2007 14:51:54 GMT

We attempted to use EZsort on a recent project and found that we were unable to draw conclusions from the output. My experience with drawing people to consensus within a room has been far more successful, and you get immediate buy-in from everyone.

—Marsha Waren,
University Information Technology Services
Indiana University

dvd — Fri, 26 Jan 2007 14:51:54 GMT

I am so glad to have found this article, and additionally, the concept.

This will be a great help in addressing the shortcomings of sites which seem to confuse rather than inform users.

Jacob — Fri, 26 Jan 2007 14:51:54 GMT

First of all thank you for your answers Todd, and fast answers I must say – what a service!

I could infact also use some specific examples of how the cards could look like, both the front and the back.

Is there any chance you could find an old card and type in waht it says, or maybe even better scan/photograph it?

Thanks again.

Michael — Thu, 15 Feb 2007 19:41:26 GMT

EZSort, a free tool from IBM uses statistical cluster analysis to help visualize sorting patterns on a sliding scale of affinity . . what a mouthful I know. but try the tool. It’s FREE, a little clunky, but is a great aid in visualizing card sorting data from multiple subjects. http://www-3.ibm.com/ibm/easy/eou_ext.nsf/Publish/410

Todd Warfel — Thu, 10 May 2007 18:32:14 GMT

The recommended setup is to have one group of three in the room at a time accompanied by the moderator. Having more than one group distracts them from the task at hand.

If the stakeholders want to observe, it’s best to have them do so behind a one-way mirror. Alternatively, we’ve used a large conference/ball room before to allow the stakeholders observe at a distance. Their reaction is interesting – you get to see the lightbulb go off in their head once they see the participants actually performing the task. It helps with buy-in to the technique as well. Just make sure you communicate with them upfront that they’re there to observe and should remain still and quite.

As to having the board participate, we’ve also had good experience with this in the past. One of our clients questioned the reliability of the technique prior to seeing the results. So, we ran them through the same exercise, quickly mapped out their results then presented their results with the analyzed results from the five participant groups and showed them how 85-90% of their input overlapped with the participants’. They were pretty amazed and we got instant buy-in.

I would caution you against putting any of the board members in with target groups at this stage as this takes considerably more moderation to make sure the board members don’t outweigh the target group. You really shouldn’t try something like that until you’ve gotten the technique down and have practiced the moderation.

Good moderation is one of the keys to a successful, balanced, representative result.

Jacob — Fri, 26 Jan 2007 14:51:54 GMT

A truly great article, I just have two questions after reading it.

Doing card sorting group-wise, should the groups (consisting of 3 persons) do the card sorting while being the only group in the room, or could you have all five groups in the room at the same time? I am figuring you can only have one seeing as how you need to note all these data down.

Second, I am thinking of doing this for a project I am working on.. the local commerce organization is having their website re-designed by the company I work for, and a card sort would be great here. I am wondering if the board of this organization also could participate alongside with their target group?

Alex — Fri, 26 Jan 2007 14:51:54 GMT

sorry for multiple postings. This form is throwing 404 messages.

Alex — Fri, 26 Jan 2007 14:51:54 GMT

this article has just been translated into Russian and published at http://www.webmascon.com/topics/planning/19a.asp

Alex — Thu, 15 Feb 2007 19:38:43 GMT

this article has just been translated into Russian and published at http://www.webmascon.com/topics/planning/19a.asp

Alex — Fri, 26 Jan 2007 14:51:54 GMT

this article has just been translated into Russian and published at http://www.webmascon.com/topics/planning/19a.asp

Daniel Szuc — Tue, 06 Feb 2007 11:34:34 GMT

Super article!

Other questions we have asked during card sorts include:
- What terms do customers find confusing?
- Are there any items be included in more than one section?
- What action would you take if you could not find the information you need here?
- What items and groupings were easy to create?
- What items and groupings were difficult to create?

We also find we can run a quick interview prior to the card sort to elicit some information that may assist us understand the types of information people are looking for on the web site.