Comments on Research Logistics

Jon Innes — Wed, 06 Apr 2011 23:51:24 GMT

Good article Demetrius, now I’ll have a handy reference to explain this stuff to user research newbies.

One thing I’d add to your list. I always ask that the researcher document the details about any participants the findings are based on.

Always ask for the screener questions and responses as part of the final report. If the tasks and the participants are not correct, nothing else matters. A well moderated and documented study based on the wrong tasks or participants is worse than having no data at all. It can be highly misleading.

I once investigated a study at a major company that was completely flawed. Not only did the company waste time and money on the study. The final loss was probably in the millions, once you factored in all the resources spent on following invalid recommendations, and the impact on the business overall.

George Foreman — Fri, 02 Jul 2010 09:31:10 GMT

Interesting insights and tips can be found here. It gave me some nice ideas that I am hoping to implement soon, so I thank you for sharing it.

Aida Cortes Lopez — Fri, 12 Mar 2010 14:24:50 GMT

I also think this is a good list of things we have to communicate to our clients/ project managers…

You call your article “Research Logistics. A Crash Course for Designers and Stakeholders”
I would like to point that many designers today get research competance at school, if not all? Do you have experience with designers who are not well known with the things you highlight?

Best ragards,
Aida

Seth Gordon — Wed, 20 Jan 2010 18:20:58 GMT

If this is for the person planning the test, here are a few additional suggestions and tips.

Arrange 1-2 pre-screened floaters to hang out in the lobby in case there is a no show. These people are on-call and ready to jump in if there’s a cancellation or if your tests run quicker than expected. If you have a room full of observers ready to go, their time isn’t wasted by a canceled session.

Send all test respondents a thorough e-mail detailing the arrival time, location (with map), who to ask for and where to go when they arrive, your contact info if they need to cancel, suggested attire (people tend to overdress because they treat it like a formal interview). I also make it clear that people 15 minutes late will be turned away.

Call to confirm the day before the test.

I was able to get recruiting of 10 respondents down to 3 days. I posted a listing on craigslist, in the Etc section, which linked to a surveymonkey form with of about 10-15 qualifying questions. in SF / NYC / LA / Seattle, i’d have at least 100 completed surveys in 24 hours.

You can argue about a bias in CL candidates, but with some good filtering and phone screening, I always found the respondents to be an articulate cross-section of my target audience, which was always web/technology, which CL people index highly for.  and, it beat paying an agency $150/head to recruit the same kind of people.  If you schedule morning / after-work sessions, that helps pull in a wider variety of respondents, including people who have full-time day jobs.

happy researching

-Seth

Demetrius Madrigal — Tue, 05 Jan 2010 21:38:09 GMT

I’m glad that you found it useful Mark. I was hoping that it would provide a useful reference even for those that were aware of the research process to help inform those that they work with.

Mark Waldo — Mon, 04 Jan 2010 20:39:20 GMT

Looks good. This article is a nice summary for clients. I could easily give this to my current client; it captures our next couple of months of testing (usability and day-in-the-life journaling). Thanks,Demetrius.