James Green — Thu, 14 Jun 2007 21:59:40 GMT

This sounds like a good idea. I also have never seen the topic elsewhere.

When I conduct user tests, I usually ask the person to note for me the point that they would “give
up” were this task being completed in the “real world” and not in the lab. This gives me real
performance data that better resembles what users on a live system might experience.

I fully agree there can be great value in having them keep going past that point or even just let
them struggle on and on. I have found that while there can be discomfort as you said, you can get
a lot of good data you would otherwise miss if you let participants go with their assumption that
the only goal is success on each task despite your introduction spiel to the contrary.

This last point may or may not be relevant to your article, but I also struggle with the issue of
stopping the task when my clients ask for “average time to complete tasks” as you’re always
thinking, “if I give them a few more seconds they may get it, and that becomes a data point” vs.
“let’s stop their misery and move on to the next question. Sorry Client, there was no average time
to complete task, they never figured it out….”

Anyway, I look forward to reading the article.

Boxes and Arrows: Comments by James Green