Jeff English — Fri, 26 Jan 2007 14:51:56 GMT

Hi David
We are not using this technique for evaluative purposes – it is very exploratory in nature. This technique helps us to understand the software customizations made to suit business needs and to find out how people accomplish tasks with it. We are not using this technique for design validation. The use of the word “usability” in the earlier post was a separate point to describe what we might expect to get out of relationships with customers that have deep customizations, without regard to the techniques available. Sorry for the confusion!

Jeff English — Fri, 26 Jan 2007 14:51:56 GMT

Hi David
This technique can only provide as much context as can be “observed” through a phone line and web conference. One way of collecting additional information is to ask the participant to send us digital pictures of their environment and themselves. We also ask for training documentation and resources available to users that can help us understand the instructions necessary for users to become productive quickly.

To your second question, the customizations that we typically see are those of adding and removing UI elements – not wholesale changes that completely change the product. An interesting finding is that our products are used in ways that in some cases are much different than our teams have expected aside from their customizations. For example, we have identified unexpected work-arounds that conflict with what we thought people could accomplish with the product. We would consider customers that make deep customizations to be interesting for identifying strategic product opportunities (perhaps for a new product or service, or one specific to an industry), but would not necessarily look to learn about usability issues with them.
—Jeff & Lynn

ji kim — Fri, 26 Jan 2007 14:51:55 GMT

Thank you so much for your article!

As you stated, remote contextual inquiry is a real valuable tool. With usability budgets getting cut and not so great economy, everybody should really think about using remote testing or conferencing softwares as part of their arsenals.

In my current and past positions designing enterprise business software, we used remote usability testing/CI to supplement our UI design and user research.
Key word here is supplement: It should be used in conjunction with traditional CI (Contextual Inquiry) methods – meaning, actually visiting customers at their work. This is especially important for new designers or researchers with limited real world experience.
Also in enterprise software world, especially in supply chain/CRM/Portal/etc., there are multiple types of users with different levels of experience. As your article stated, it’s important to really identify who your target users will be…. Business Analysts can help you identify the users – but not always. I often find myself helping my customers really identify their workflow – both in government and private industries. In past, through series of interviews and site visits, my team started to see some workflow patterns that even business analysts (customer side) or my company’s product managers couldn’t really figure out. Identifying product usage through workflow can be pretty valuable in enterprise softwares. So do this first – well at least some of it. After awhile, you can see some base patterns that can be categorized by customer’s vertical industry, size, culture, deployment, etc.

Remote usability testing/CI again is a really useful tool. But in the end, just my personal opinion, it will only help you identify some GUI level problems – e.g. labeling, basic interaction,etc. Changing one area of the UI can lead to problems for other users (for programmers, similar analogy is fixing one bug sometimes leads to addional bugs…). Unlike consumer softwares, it’s very hard to figure correct workflow and design intuitive UI for each customer in enterprise softwares. Then again, these customers are paying lot of money for your enterprise software. My recommendation to decisions makers: hire some good UI people in your professional services team.

Some simple recommendations for conducting remote CI:
1. This is basic, but often left out. Print out time zone for different parts of the world, or have access to time zone converter application.
2. If you are using webEX or similar software/services, try following setup: computer that you will use to connect with your user remotely, and have another computer connected also to conferencing session. But have the other computer record the session. This is more reliable method – since recording screen sessions can be CPU intensive (might crash your computer).
3. From my personal experience, having session over hour can be tiring for the users. Unless your users has a webcam, it’s hard to tell if your user is getting tired…..you tend to get fuzzy responses then your users get tired :)
4.I don’t work for camtasia, but recording the session over camtasia like software over webconferecing software is recommended. I don’t recommend using NetMeeting for various reasons – I just found it easier to get people to use webex or placeware.
5. Email some pre-interview questions in advance. I also usually add a link in the email to get users to test if their computer will work with remote conferencing software.
6. Expect about 10 -15 minute delay when you start the remote CI session – often it will take users 10-15 minutes to get their computers configured to use web conferencing services…....

Of course get permission to record the session and etc..

Comments on Remote Contextual Inquiry: A Technique to Improve Enterprise Software