Comments on Investing in Usability: Testing versus Training

dawn marie — Thu, 05 Apr 2007 17:38:31 GMT

If I had $10,000 I would keep usability fully in the project.Don’t succumb to “either/or” statements, but how can we get user input AND remain cost effective.

You see, you can have testing early and cheap. Use paper prototyping ahead of development for testing. Walk down the hallway of your building and ask people to walk through the pencil sketch, wireframe or mockup. Ask questions like, “what would you do first if you needed to enter your expense report, what stands out first”, etc. Granted this is very informal, but you can gain an understanding of the UI by asking 15 people – 5 minutes a piece. If my dev team needs an answer today, I can get very close to accurate in 75 minutes.

Also, I use UT to show our success rates for CBA – not only for enhancements for the next release, etc. I am aware the user groups may not be involved, but the intuitiveness of the product can be achieved in this manner.

Lastly, training over testing can be very expensive. Our accounting company is one of the largest in the nation/world with mulitple offices in every state of the nation and abroad. We have a large amount of new hires who are here seasonally – just out of college for their first big job. Can you imagine the training dollars? In this case, it would be well over $10,000.

Alok Jain — Thu, 05 Apr 2007 17:21:04 GMT

I think it depends on what is the problem you are trying to solve. The Article seems to assume every interface has same goals but Usability itself is not absolute. There are trade-offs in usability and the biggest one I believe is efficiency v/s intuitiveness.

Mostly usability tends to get associated with intuitiveness, which I disagree with. It has t be based on user goals

Let me take an example, if you are building an application a call center, the user goal (tied to their appraisal) is to complete max # of calls per unit of time. This task involves several complex sub tasks like checking user’s profile (consisting of 20 different attributes) finding right information based on user’s call and profile etc etc..

The goal is efficiency. In such scenario it is fine to let intuitiveness take a back seat and can be supported with training.

But if the base goal of system requires greater intuitiveness then there is no point investing in training alone.

I agree that budget constraints would require alterations to process and finding more efficient mechanisms but conclusion that training is better would not be the right one.

Gene Averett — Tue, 28 Aug 2007 00:25:36 GMT

I think what John is trying to point out is that with a limited budget and his experience with usability professionals he would prefer to have the usability people provide insite to the designers/developers. There is some truth to what he says about usability people not being able to design, most come from a science background and have read all the literature about usability, but very few can design and this is a major limitation to the usability environment. As people adapt to the new technologies usability should also adapt to the abilities of these users. We can not treat everyone as an novice and design for that, there needs to be a middle ground. Usability needs to evolve as the users and technology evolves.

Dave — Fri, 26 Jan 2007 14:52:13 GMT

Hiya John,

I think I agree w/ your overall statement. That if I have $10k to put into usability testing, wouldn’t that be better spent on training designers to be better at doing self analysis. I tend to agree with this sentiment a lot.

What others have said about usability being more than just testing to me obviously ignores your major premise. There is nothing here that doesn’t state that there can’t be more to what the designer does (it is a human being after all, w/ the ability to do many roles; so the education can just keep continuing.)

The one area that I feel where this doesn’t work is when the complexity of the solution reaches a critical level. I can’t quanitify what that level is, but I can say that I beleive that when complexity reaches a certain level another central nervous system that is more experienced in specifically evaluating design is required, that is separate from the formation and generative processes. Sometimes you just need another body with a different POV.

mhendrie — Mon, 16 Apr 2007 09:22:10 GMT

Whilst John’s article clarifies that “Usability” is both a process and a set of tools, he mistakenly implies that every user experience designer, usability specialist and interaction designer thinks of usability as a “test’. Apart from points #8 and #9 the authos reasoning ignores the enormous amount of work done in heuristics, contextual inquiry, simulation and prototyping sessions as well as styelguide development and team wide training practices. I’m glad that he has worked with designers with good usability training but to oppose Testing versus Training is a false problem, as though they are the only two variables in developing a good User Centered Design. Surely it should always include iteration, prototype testing and user feedback? Any solution proposed without developers on the team is bound to be shoddy. Hint: dot work with siloed teams. Rather thanspoon feedin designers many PMs could better help their teams by nailing down real user requirements, use cases and noting that many software maintenance costs come from unmet requirements rather than uncreative usability specialists.

Daniel Szuc — Thu, 05 Apr 2007 17:21:16 GMT

John raises some great points.

The importance of asking the right user research questions and helping Product Managers to define/design products in the right direction that helps end users and the business make more monies.

How to bridge usability testing and user research data into designs that impact the business positively. Otten a huge gap can appear between the the Usability testing data and how this data can be used to drive the design more effectively.

Suggest there is also an wonderful opportunity for usability folks to move over time from a tools (bottom up approach) to assiting drive products strategically (top down approach). As we plan to pass on the knowledge and tools to those who need it most.