Boxes And Arrows : The Design Behind the Design

Using Site Evaluations to Communicate with Clients

by Dorelle Rabinowitz on 2003/01/13 | [12 Comments]

How do you prove your worth to clients in today’s difficult economy? One of the tools in my arsenal that has proved tremendously effective is a website evaluation (or assessment). Performed as part of a sales proposal, a site assessment can help you speak knowledgeably about solutions to your potential client’s problems. As part of a “discovery” phase of a project, it can help uncover opportunities for improvement. Additionally, it can serve as a benchmark to be tested against later in the design process. Because many clients understand ratings, site assessments early in a project can help you and your clients speak the same language, establishing a base vocabulary you can reference later when you do user research, personas, card sorts, and usability tests.

Over the last few years and umpteen site evaluations, I’ve developed some templates I customize as needed. I will describe these templates, when and why I use them, how I present the assessments, and will discuss some specific cases where site assessments have helped me convey my arguments to clients. If you already know what an assessment is, move on to the Three basic templates section. Otherwise, a brief introduction follows.

What’s an assessment?

Part of a spreadsheet that shows categories and their attributes, along with scoring and comments. (click to enlarge)

Two kinds of assessments
A current state assessment evaluates one site, typically the site you are redesigning. I use these assessments to help define problem areas as well as successes.

A competitive assessment is a bit different. This type of assessment helps set an industry “marker” by looking at what the competition is up to, what features and functionalities are standard, and how others have solved the same problems you might be tasked with. Depending on particulars, it’s a look at a bunch of sites, none of which you are designing.

Tools
I start with an Excel spreadsheet divided into categories. Each category has a number of attributes underneath, questions that the evaluator must answer. Each attribute receives a score, and scores are averaged to come up with a total for the category. Since scoring is so subjective, I make specific comments to argue my point of view. If a point is particularly engaging (in a positive or negative light), I pull a screenshot that illustrates it.

Rating scale
I like to use a scale of 1 to 5 (No capability=1, Average capability=3, Best practices=5), but you can also use a high/medium/low scale. And sometimes numbers seem too set in stone, or are bound to cause controversy with your client. In those cases, I use Harvey balls, with an empty ball signifying no capability and a filled-in ball best practice.

Perspective
Decide on your point of view and this will help determine your category headings. Are you evaluating only information architecture? Then your categories would be subsets of IA, such as navigation, taxonomy, and information organization. Are you looking at a particular feature? Your categories would reflect how that feature functions.

Notes on adapting
It’s also worth thinking about why you should do this analysis. What are your strengths? Where does your expertise lie? And, concurrently, what kind of results do you anticipate? Has the client come to you with regard to a particular problem?

Three basic templates

I start with three different base templates, and modify them depending on the project and on my role. I customize each template depending on the project, sometimes down to very specific attributes. For example, on a financial services assessment, I added macro-level questions about specific content items I knew were required: real-time quotes, interactive chart features, etc. For the same assessment, I also looked at features the client specifically wanted us to improve, such as search, personalization, and customer service.

Template 1: IA
My IA template looks closely at information architecture/interaction design details. It really picks at the nitty gritty details of navigation, classification, and interface. Remember all those “rules” we’ve struggled for so long to practice? These rules are my attributes. Using this template is a forum for showing my clients my main area of expertise. It also helps them see their site (or their competitors’ sites) from my point of view.

For example, some categories and attributes are:

Information organization

• Is information presented in a logical structure?
• Is it clear how site components and sections are related? Are content/function areas logically/intuitively grouped?

Navigation

• Is the navigation consistent across the site?
• Does it help users answer the three fundamental navigation questions: Where am I? Where have I been? Where can I go?

Nomenclature

• Is terminology clear and consistent across the site (including navigation elements)?
• Does the site avoid the use of incomprehensible industry jargon?

User interface

• Does the site support both experienced and novice users effectively?
• Are visual cues are easily understood and consistent?

Using this base template helps me offer my clients solutions to their specific problems. If the navigation is inconsistent, I can describe the benefits of persistent global links. Combined with other analytical tools such as log files, this evaluation can inform a sales proposal to improve the client’s IA, and can also help my company create a very specific statement of work.

Template 2: Proposal

When I am performing an assessment as part of a proposal effort, however, I often expand the IA assessment and evaluate from the larger customer experience point of view. The assessment forms the basis for my company’s arguments of which features need to be enhanced, which features need more emphasis, etc. This template starts with the IA categories and includes some important others.

For example:

Content strategy

• Is the content relevant to the target audience?
• Is the content up-to-date?

Brand

• Is the brand identity clearly presented?
• Is the brand applied consistently across the site?

Visual design

• Is there a consistent use of iconography and visual design language?
• Are visual elements used effectively to show relative importance of various screen elements? Do the most important things stand out?

I have successfully used this template for large enterprise clients in financial services, media, retail, and healthcare, as well as smaller, more unique clients such as museums. Because much of my work is a team effort, this assessment helps clarify why there are so many different people on this team, and how we are all interdependent. And the assessment’s worth can be increased even more by adding technical and strategic categories, helping my company prove our value even more. No matter how inclusive the assessment becomes, the message is always the same: as a user, here’s what I see, and here’s how my team can improve the customer experience.

Template 3: Lifecycle
Another approach that I’ve found to be powerful with clients takes a different perspective, by changing the focus of the assessment. By using the lifecycle of customer experience—Attraction, Orientation, Interaction, Retention, Advocacy—as your categories, you can apply your own vision and opinion within a concrete framework. This template is the most “consultanty” of all, but I’ve found that when clients aren’t able to articulate their goals, this assessment can help direct the conversation. It’s also the most holistic view of an assessment, viewing a site as an “experience.”

Attraction

• Is my attention drawn by my initial experience, and do I feel welcomed and familiar?

Orientation

• Can I quickly understand what I can do here, and do I feel invited to look at different options?

Interaction

• Does the content of the site engage me? Is it possible to discover new information?

Retention

• Does the site offer a way to help me know when special offers are coming? Is there a way gain from other people’s experiences? Does it link to complementary offerings?

Advocacy

• What is the site doing to keep me coming back? What is the site doing to encourage me to share my experience with others?

Some examples
A current client asked my company to bid on a project for a digital photography site geared toward teenage girls. To choose which sites to evaluate, I broke the assessments into three tiers: business competitors, functionality competitors, and demographic interests. Since the client was already familiar with our team, I didn’t need to establish my competency with user-centered design principles, and since they were looking primarily for our point of view, I decided to make template 3, Lifecycle, my starting point. I added high-level views from the other two templates and customized some categories according to functionality.

Fig. 1: An assessment captured at a glance that uses Harvey balls for scoring and short text summaries. (click to enlarge)

Then I assessed about five sites about photography—looking particularly at the image upload features. I also checked out six community sites for teenage girls, as well as a few sites that already combined community and photography. Overall I looked at about 15 sites. I already knew that to engage this client, the pace needed to be quick and the visuals needed to be showy, so I kept the comments short, sweet, positive, and made liberal use of screenshots (see Figures 1 and 2). Using these assessments, I was able to describe successes and failures succinctly, helping to complete our client’s vision and goals for the new project.

Fig. 2: Screenshots help illustrate important points, good and bad. (click to enlarge)

Another time, as part of a current state market evaluation, I looked at ten different financial services sites, evaluating everything from features and functionality to branding. This client wanted cold hard facts and needed to prove the cost benefits of our project to his superiors. So this assessment used numbers to rank features, with each site receiving an average score per reviewed category. The assessment accomplished two things: first, it reinforced our client’s view of the current marketplace and competition; and second, it pointed out a hole in a competitor’s online offering, an opportunity our client thought they could leverage to get ahead in the market.

Time spent on assessments varies depending on the complexity of the site and how deep the assessment is. Typically I spend an hour at a minimum; however, I can spend a whole day doing a really involved assessment. For fairness, when comparing multiple sites for a project I try to spend the same amount of time on each.

Notes on presenting, or How to bring the message home

Fig. 3: In this example we balance our unflattering critique with positive suggestions for improvement. (click to enlarge)

In many cases clients are impressed with a spreadsheet full of data. In other cases it’s necessary to add some punch—to tell it like it really is. In these situations, I use screenshots with callouts to demonstrate both good and bad examples (which is especially valuable during a client presentation or as part of a proposal). It’s all about what story you want to tell (see Figure 3).

I like to start off with something good. Look at something that works really well, and point it out. Then I can start in on problems. Or split up your conclusions into a few main points, and start with a summary page. Think about effectively illustrating your arguments with visual examples, and use your screenshots. Keep your language positive, and focus on solutions to the problems.

Last thoughts
Keep in mind that an assessment is a truly adaptable tool. And it’s not limited to IAs. A business analyst can look at how successful the site is in reaching ROI goals. A brand strategist can see how the messaging performs. A team can assess a site at the beginning of a project (averaging everyone’s scores), and at the end to see what’s improved. Remember, though, an assessment is just one part of the toolkit. It needs to be used in conjunction with everything else: user research, personas, flows, etc. If a site scores poorly on a feature in an assessment, for example, user research on the subject can uncover whether that feature is important to users or not.