Comments on Beyond cardsorting: Free-listing methods to explore user categorizations

B. Raj Giri — Thu, 19 Jul 2007 23:28:59 GMT

Hi Rashmi, It was interested to read your article. In fact, I want to know more about free-listing as I am thinking of using it for my qualitative analysis of fieldwork data. I have not found a good article talking about freelisting. Can you suggest something? B. Raj (http://www.nepalsathi.ws/)

John Deal — Fri, 26 Jan 2007 14:51:04 GMT

Dear Friends,
This is the right media to extend our help full services. It’s a right step towards aid to human kind.
TRUSTPHARMA.com is dedicated to providing cutting-edge medicine through Internet technology. The Trustpharma.com physicians specialize in consulting with privacy-minded individuals and prescribing popular medications. These medications, sometimes called “embarrassment” drugs, are prescribed and dispensed by our physicians and pharmacists!
Visit TRUSTPHARMA.com, your online solution for quality generic and brand name medications at affordable prices! We have some great medications to offer you and will be adding many more in the coming weeks.
-Guaranteed Lowest Prices
-Orders Received 10 or less!
-Discreet and Confidential Packaging
-Licensed Physicians & Pharmacist
-Quality Generic Medications with FDA
-Who and GMP approval

Jane Perez :)

Bob Huerster — Fri, 26 Jan 2007 14:51:04 GMT

I was recently introduced to the game “Taboo.” The object is to get your team mates to say a word without mentioning certain “taboo” words to give them clues. To take a simple example, if you wanted to make them say “birthday” you might not be able to use the words “happy” or “cake” as clues. Coming up with clues is quite a challenge since the best words always seem to be “taboo.” I wonder if the developers of the game used methods similar to those discussed in this interesting article.

Larry Wood — Fri, 26 Jan 2007 14:51:03 GMT

Sorry, I’m a bit late joining this. First, let me say a very provocative article for one who has recently decided to make card sorting techniques and tools a serious research endeavor. First, could you be a bit more specific about the method? As a cognitive psychologist, I’m generally familiar with the concepts and notions here.

Is this just a free association task? It seems to me that this is a classical situation where instructions make a huge difference. Rashmi, could you share a brief overview of one or two studies you’ve conducted, using Free Listing. Or do you have any reports you could share off-line via email?

rashmi — Fri, 26 Jan 2007 14:51:03 GMT

Thats a tough one Peter. I spent quite some time thinking about this issue. On the one hand, free-listing can be used to enumerate items at any level in the hierarchy (so asking about either “websites” or “types of websites” is fine). But faced with a choice, I would go with “list all the websites” and then doing a card sort on the list that is generated. Research on free-listing seems to indicate that free-listing works very well when one is listing specific items. CNN.com and Google.com are specific sites that one has experience with, whereas Newssites or Search Engines are abstract categories. I think that free-listing will work better for specific items rather than categories. (My reasoning goes back to the differentiation between implicit and explicit memory. Explicit memory is tied to specific episodes and free recall tasks will tend to bring up such (explicit) memories. For example, when you think of the websites you have used for the past few days, you might remember making reservations on Expedia, reading about international politics on BBC.com and checking reviews of a digital camera on Epinions. This seems more likely than remembering making reservations on some travel site, checking international politics on some news site, or doing finding our more about a digital camera from a review site.)

Why does “list all the types of websites” seem like a better approach to you?

PeterV — Fri, 26 Jan 2007 14:51:03 GMT

I was thinking: to understand how they look at the domain “the web”, you could ask “List all the websites you know”. Or you could ask (and this feels like a better approach): “List all the types of websites you know”. I’m not sure. Any ideas?

rashmi — Fri, 26 Jan 2007 14:51:03 GMT

good point. IMO, using the same group of participants for both the free-listing and the card sorting exercise should not be a problem. A free-listing exercise will remind the user about their categorizations, but should not impact their categorizations in a way that will negatively impact the card sorting exercise. The reason being that categories and concepts develop over a period of time and are relatively stable.

But I do think that it might be difficult to do both free-listing and cardsorting in one session. If the purpose of the free-listing exercise is to generate items for cardsorting, then you need to finalize the list of items before beginning the cardsorting exercise. One possibility is to conduct the free-listing exercise via email, and ask the same participants to subsequently come in for the card sorting exercise. This would have the added advantage of letting you screen participants according to domain knowledge or some other criterion based on their free-listing data.

I wonder if one could conduct a group free-listing exercise. Ask a group of people, sitting around a table, to each write down a list of items in a domain. Subsequently, each person should read their list aloud their list. The second person should only read items not mentioned by the first person, and so on. In this way, one will compile one long list in the process of going around the table.

I have not tried this, but think it might work. What do you think?

Blake — Fri, 26 Jan 2007 14:51:03 GMT

I wonder if it would help or harm a cardsorting exercise to first conduct a free-listing exercise.

It is beneficial to have both free association and prompted association data, and to get it from the same participant, which saves money and time. I wonder if it would introduce a significant bias for the second, card-sorting portion of an activity.

Thoughts?

PeterV — Fri, 26 Jan 2007 14:51:03 GMT

All I can say is: brilliant. There are a whole bunch of research methods out there waiting to be discovered, adapted and put to use by the information architecture community. Excellent article!