Comments on Real Wireframes Get Real Results

Dawn Nidy — Fri, 26 Jan 2007 14:53:19 GMT

I agree with the author that if you are trying to get early feedback on paper, it’s generally a good idea to add fidelity and visual cues. I also like the idea of testing HTML prototypes, but it’s often much faster to get it out on and tested on paper. If you’re using templating in Visio, it will be very easy to drop in navigation, logos, pictures, etc. to make the wireframe more useful.

Absolutely you should be collaborating with your visual designers and copywriters before letting anything go out to usability testing. Interaction design is only one piece of the puzzle and not having those in will skew your results.

I am not convinced you need color as part of your wireframes for your internal audience unless the interaction designer is also responsible for the visual design. Or, if the visual designer is going to add their design to the wireframe to create a common deliverable. I think this depends on the project, organization and team as to how appropriate this is.

Max Lord — Fri, 26 Jan 2007 14:53:18 GMT

I have had a contrary experience to the author. Often a wireframe can be *easier* to use than the finished site, especially if the visual design is not so great.

I find that navigation and form elements tend to be the only colored items on wireframes, and that makes them easier to find. Often, they are much less obvious in their final state. I have asked visual designers to squint at their comps and make sure that the “colored stuff” in the wires is still popping.

Venkat Venkat — Fri, 26 Jan 2007 14:53:17 GMT

It’s true. The quality of the prototype affects the feedback you receive, but testing early on with lesser fidelity prototypes can be really valuable

Abdul Kaleem — Fri, 26 Jan 2007 14:53:17 GMT

yeah!! its definately a good suggestion by Stephen Turbek spending 10-15 mnts more on wireFrames and comming up with actual content and visual design.

Theresa Cunnington — Fri, 26 Jan 2007 14:53:17 GMT

I believe it is up to the facilitator to manage participant expectations when testing with wireframes. I recently tested wireframe content pages with a fully designed home page layout with great success.

I’ve tested with prototypes as basic as sketches on flipchart paper and created flyout menus with sticky notes and still received rich, valuable and useful feedback.

Baldly telling them not to focus on the design usually fails, but telling them that they are looking at the equivalent of a blueprint generally helps. I want them to find the bathroom for me, not comment on the lack of curtains.

Putting in dummy images tends to generate comments about the image and whether the participant likes it or not. Putting in a box with a comment inside that says ‘Image’ lets the participant assume that they will see a picture there, without them needing to process it.

As mentioned, you want to avoid lorum ipsum and use real text and I always visually distinguish links and buttons so they look clickable. Yes, there has been the odd participant who’s commented that they don’t like the ‘grey’, but that just indicates to me that I need to stop the test, manage the participant’s expectations again and then carry on, not that testing with simple wireframes is a bad idea.

In addition to Ben’s comments above, the problem with electronic prototypes is that the participant expects the entire prototype to work. They are constantly clicking buttons that don’t work and get a negative perception because the ‘site’ appears to be ‘broken’ all the time. Testing on wireframes manages their expectations. They know it isn’t finished and have lower expectations of what they can click on.

Testing with wireframes is an excellent way to obtain feedback on navigation, labelling, workflow and ease of use. Make sure they can find the bathroom first, then start decorating.

Charlie Nichols — Fri, 26 Jan 2007 14:53:16 GMT

Good stuff.
I’ve used what I call “frankensteins,” part wireframe, part real page elements for a while. Similar to what Steven points out,
assuming we are designing new areas for an existing site, we can use real headers, real images and even real visuals of form elements/text. I also agree with real explaination text, as lorem ipsum doesn’t transmit enough information to business users.

Zef Fugaz’s above comments are good too, these can be another step in different documentation that delivers to different needs. I’ve had to deliver upwards of 4 different “site/content maps” to a client, all valid “site/content maps” but all showing different perspectives of the data at hand. We sent different docs to different stakeholders that had different priorities.

Stephen Chalastra — Fri, 26 Jan 2007 14:53:16 GMT

Wireframes are a tool. A screwdriver is a tool. And just as it is not very efficient to knock in a nail with a screwdriver, there’s not much point using a Robertson screwdriver on a Phillips screw.

I work for a software consulting company and am faced with a wide variety of application and user types. It might occasionally be useful to have higher or lower fidelity wireframes than a typical Visio mock-up, but I find this to be the exception. It really depends on the complexity of the application, the phase of the project, the extent of the differences between this and what the users are used to, and so on.

There are several types of information that we want to get from user reviews. For some applications, the appearance is a high priority, and for these we produce a visual prototype. I find standard wireframes valuable for evaluating the data semantics of the panels: the field labels, priorities, groupings of data, and so on. And I find that if we introduce too many variables, the focus of a review can be diminished.

I’ve never heard the comment “is the application also going to be in black and white?” But I have heard over and over again something like, “I don’t think this blue goes with that orange.” If users are given the proper perspective up-front, they can typically not only relate to the wireframes, but appreciate them as providing a visual focus around which to wrap a text-oriented business requirements document.

Having said this, we’re on the cusp of changing technology and are increasingly moving from paged to rich internet applications, in which static Visio wireframes are not so useful. But that’s another topic.

Austin Govella — Mon, 25 Jun 2007 15:09:43 GMT

It’s true. The quality of the prototype affects the feedback you receive, but testing early on with lesser fidelity prototypes can be really valuable. Most teams should have a set of wireframe templates that leverage existing identity, visual styles, and common assets (like custom buttons, etc.). This is only really possible if you’re extending existing products. Working from these templates, the time to make your wireframes more “real” dissappears.

However, if the problem is users don’t understand the conceptual nature of the wireframe, this is as much to do with education and culture as it does wth the wireframe’s fidelity.

At the World Bank, user testing is fairly strongly embedded in the culture, and the usability guru does an excellent job of educating users as to what an HTML wireframe is, what it isn’t, and the purpose of the specific test. Users test both bare bones HTML wireframes as well as higher-fidelity, grayscale wireframes. The higher-fidelity wireframes had to be grayscaled because users believed color versions were work that had already been completed, and there’s a bias against critiquing already completed work.

A common reason for using lower-fidelity wireframes is that users may hesitate to provide the same feedback they would if they were presented “ideas” for the site.

At the other end of the spectrum, at Comcast Interactive, I’ve heard stories where hi-fidelity prototypes were interpreted as finished products by management who liked what they saw and mandated a launch.

The same education and culture issues come into play, but getting real is certainly not without its problems.

Ideally, your wireframe should be crafted with its use in mind. Documenting decisions for history or for feedback should create a different wireframe than one used for testing or another used for communicating design with the rest of the team.

Ryan Romro — Thu, 07 Jun 2007 17:10:16 GMT

Wireframes can be used for usability tests in instances where visual comps have not yet been made or if visual comps are out of scope of the project. Usually clients will want to test wireframes as a first step to seeing how effective the information architecture is presented to the users, without being distracted from visual representation. I found this to be effective in the beginning stages to get a grasp on what significant final decisions need to be made before comps are started.

Usually the wireframes that I’ve developed are minimal in color with an exception for links and icons. I try as much as I can to avoid lorem ipsum, not just for the client but also as a benefit for our copywriters to have something to start off of. Using real information is key in any wireframes, especially when testing the structural layout of the site.

An understanding of what wireframes are should be made during the discussion of the statement of work. Knowing that visual is separate from ID/IA work is key to the client’s understanding before any work has begun and should be reminded in instances when they do go off track.

I do find it however that wireframes and visual comps clash in a way – knowing that most clients will give “LATE” feedback on ID/IA work that has been signed off. Yes – Clients suck. Well most of them at least. =)

Jonathan Baker-Bates — Wed, 07 Mar 2007 15:17:39 GMT

Until now, I wasn’t aware that putting wireframes in front of users for the purposes of usability testing was in any way common practice. It seems a very odd thing to do. No wonder the author sees it as a problem, and no wonder most of the comments here are coming back saying “That’s why we use prototypes!”

Fred Beecher — Fri, 26 Jan 2007 14:53:16 GMT

Stephen… I whole-heartedly agree with the points you make. Excellent article.

In my experience, increased use of rapid prototyping techniques makes having an intelligible prototype all the more important. I’ve used Axure for about a year, and of course I get the “it needs more colors” comment, but the most significant thing that I’ve noticed is that prototype test participants are unsure of whether they have completed a task if you use genericized page types. When I take the time to put in fake data that matches the tasks in the test, users have a lot more confidence about whether they’ve found what they’re looking for. It helps prototype tests more definitively answer the question, “is this usable?” This leads me to believe that a new rapid prototyping best practice would be to complete the prototype tasks *first* and then begin working on the prototype. This could increase rapidity even further.

Mark Kraemer — Mon, 10 Dec 2007 16:17:03 GMT

Fidelity for prototypes can be roughly measured by 3 attributes: visual detail, functional depth, and technical reuse. The obvious trade-offs are the more realistic any of these attributes becomes, the most cost there is involved. The effort for including visual detail on each page I think is discounted above when you consider the cost of maintaining that consistent style over multiple pages, unless you increase the technical reuse attribute as well (CSS) which in turn provides more cost.

I don’t think there’s a single silver bullet / best practice for how to prototype for every single project. One should consider who the prototype is for (investors? the client’s business analyst? end users in usability testing? your developers?) Each group will have different needs in each of the 3 attributes. Design your prototype to meet the needs of these interim users and you’ll find the right balance of cost/value for visual detail, functional depth, and technical reuse.

Zef Fugaz — Fri, 26 Jan 2007 14:53:15 GMT

I agree that wireframes are a technical document and that visual prototypes (containing real or sample content) are far more suitable for usability testing and client feedback. My own design team create wireframes in the first instance to establish the UI framework. Once we are happy with the framework the wireframes are then split in to two complementary models. The early wireframes evolve into ‘wireframe templates’ – these descibe the content elements/tags/metadata for each template (then used by the developers who are building a content management system or other software). The complementary model is a ‘visual prototype’ – this shows a contextural view of the wireframe, and contains actual content, labels and sample images – like the examples in your article. This system works really well for us. However, in our case we sometimes have to handover the Information Architecture to a branding or marketing agency who create the final visual designs (since we have perfectly capable visual designers in-house this constantly proves a challenge). As far as I’m concerned the Information Architecture is influenced by the visuals (colours, fonts, text sizes), so it can be quite tricky working with other design agencies who are primarily focused on the branding…

Ben Darlow — Tue, 10 Jul 2007 16:14:38 GMT

The one reason to use wireframes that isn’t merely for the benefit of the designer is that of communicating purpose to the client without distractions. It’s an interesting idea to use colour and make a pseudo-wireframe for usability testing, wireframes still have a very important role in an earlier stage of design, when proposed solutions are still being mooted.

This is especially important if the new proposed designs are very different from an existing design, as differences in colour and typography may be sufficiently apparent to the client that they end up becoming a focus, when really they have less significance than the layout of the page and UI elements. Going straight to a prototype means that early decisions about colour, branding and typography have to be made at a stage when really they shouldn’t be allowed to cloud the subject matter.

That all said, I think there is some value in having these chrome parts visible once you get to the user testing stages. Perhaps it would be an acceptable alternative to just use a generic stylesheet at this point?

David James Nicol — Fri, 13 Jul 2007 03:59:56 GMT

Interesting article.

My initial reaction is to ask why bother with wireframes at all?

With access to the right skills, it may well be easier to go straight to a HTML prototype. This, I believe, could be better at overcoming the problems external people often have with wireframes.

Going straight to a prototype would be especially applicable if the project is to improve an existing page or site (is in the example), as the current pages provide a ‘template’ which can form the basis for the prototype.