Boxes and Arrows: Comments by John Ferrara

John Ferrara — Fri, 30 Mar 2007 12:32:38 GMT

Practical rapid prototyping concept, using existing processes but taking them a little bit further. Would also like to see live examples, and perhaps a link to a video demo of a usability test using a PDF prototype.

John Ferrara — Mon, 16 Apr 2007 13:53:11 GMT

I think this would be a great topic for an article, and I’d be particularly interested in finding out how people have worked prototyping into the Agile process, i.e., does it expedite the process, does it create advantages for the UI designer, etc.

Todd, I’m a bit sketchy on the nature of the article—is it a survey, a case study, or general theory? Is the focus on trends or guidance?

John Ferrara — Mon, 25 Jun 2007 11:32:23 GMT

Responses to 2 comments above:

Chris—Glad you brought up Google Suggest, I’ve been debating whether I should include discussion of it. On the one hand, I believe that it is a borderline revolutionary assistive tool, and very much a part of the future of search. On the other hand, I intended this article to be broadly pragmatic—and suggest is a function that may be beyond the means of +90% of the readership. I actually had an overview of it in an initial draft of the article, but pulled it in a subsequent version reasoning that I could easily write a full article on suggest functions alone. Any inclusion of the topic in this article would have to be unfortunately cursory because it shares space with so many other subjects, but if there’s interest I’m more than happy to work it back in.

Dave—The perspective of the article would, I believe, be completely in line with your expectations. I think it’s uncommon that UX folks are ever the ones in charge of search, and the article is meant to provide a framework for discussing and influencing its development. I think it arose from the same sorts of concerns you’re citing.

Great comments, thanks so much!

John

John Ferrara — Mon, 25 Jun 2007 11:38:20 GMT

FYI—for some reason, a portion of my comment above is rendering with strikeout formatting; I think I accidentally used a platform-specific syntax. Please ignore the format, the struck text is meant to be read.

John

John Ferrara — Wed, 27 Jun 2007 02:09:10 GMT

Thank you everyone for a ton of great comments. I’m going to respond to each of the 5 above in order.

Mike,
I completely agree, there’s so much great thinking among enterprise search vendors these days, and their products have really amazing capabilities. Customers are demanding better tools as the volume of unstructured data is exploding within their organizations, and I think that in some cases they expect the technology itself will solve all of their problems right out of the box. But I believe that the usefulness of the product varies with how it’s implemented, maintained, and used.

Chris,
Yeah, that’s a really important question: how the results display should be influenced by the characteristics of the query. I think that the conventional list of algorithmically determined results is just a starting point, and there’s a lot of value in shaping the results to the user’s needs. I’m planning to cover some of this in the article.

Ed,
As it happens, I’m currently doing some work with indexing of microformats and repurposing them for results display, as well as custom search operators. But this stuff definitely falls at the “way advanced” end of the design spectrum, and easily merits its own article. In a general article like this, the available space is going to be pretty tight. If there’s enough interest out there in these subjects, I might submit a separate proposal.

Matt,
Many of the strategies for improving content internally would indeed also benefit external search rankings. Of course indexing, query formulation, and results display would not, and there’s much more to external search engine optimization.

Amber,
Totally. I think that, in general, the quality of results from enterprise search engines compares very unfavorably to that of external engines. Of course it should be just the opposite, since organizations have the opportunity to finely tune and customize their searches to both the constraints of the data and the needs of their users. The problem is that most simply don’t.

Thanks again, this is all extremely helpful!

John

John Ferrara — Mon, 02 Jul 2007 19:24:44 GMT

Celeste,

Caught your presentation at the Summit, and it sparked a concept for a related exercise I’m proposing within my organization. I would love to see this written up in an article, and have two questions that I’d like to request it address:

1. Working from a seed sort seems to put a lot of agenda-setting power in the hands of the first person. How can we mitigate that person’s influence on the final sort?

2. Some participants may be predisposed to make substantial changes, while others may be predisposed to change very little. Taking into account this personal variability, how can we reliably determine when no further changes need to be made?

John Ferrara — Sat, 11 Aug 2007 13:59:43 GMT

Great article Kyle, thanks for writing this up.

I think that the greatest advantages you’ve pointed out are the ability to go directly from your wireframing application of choice to PDF prototype without intervening development, and then to use those prototypes in remote testing. What’s great about both is that they’re significant efficiency gains, allowing the same amount of production in less time and with fewer resources.

I have one question—when you translate the wireframes into PDF’s, do you usually make all of the links and navigation on the page active, or select narrower functional paths? Does your practice vary from case to case?

John Ferrara — Thu, 18 Oct 2007 20:38:08 GMT

Richard,

Yes, we have been able to map problems in search instances to the specific variables discussed above.

For example, earlier this year we conducted a search log analysis to evaluate the quality of results returned for the most common searches. A large number of queries needed to be discarded because we couldn’t figure out what the user wanted to find (the phrasings were commonly vague or oblique). That’s a user input problem; it would be unreasonable to expect that a computer would interpret the user’s intentions better than a human being could.

For queries where we were confident we understood what the user wanted, large numbers of seemingly irrelevant results were very common. The causes vary from query to query and result to result, and involve a complex interplay of content, index, and engine. But in many cases, a significant contributing factor could be found: the page had no metadata, the navigation was indexed, and so on. Again, any shortcomings of the engine cannot be easily resolved without disrupting well-performing queries, so the problem really has to be seen as one of the implentation.

Problems in the results display are easily observed in user testing. I’ve frequently seen the search engine return the right result, but the user skips over it because the meta description has no relationship to the query. It’s terribly tragic, because everything has performed exactly as it should but the user has no reasonable basis for knowing it.

I’ve seen these and other issues across a large set of evaluations, and they consistently map to one or more of the five variables.

John Ferrara — Thu, 18 Oct 2007 20:38:56 GMT

James,

I imagine you’re referring to the section on ontology, and you’re quite right that it’s worth stating the task is nontrivial. That said, there are engine products that offer prebuilt upper and domain-specific ontologies, as well as a growing number of independently built resources available for purchase or public use. Modifying existing work would be much easier than starting from scratch.

John Ferrara — Tue, 25 Sep 2007 13:15:45 GMT

Steve,

I’d be interested in particular in how a large corporation can effect change. Big, complex businesses traditionally benefit from the reliability of tried and true processes, but the pace of web technology favors organizations that are nimble and adaptive. How can a large organization be responsive while minimizing the risks of instability?

John Ferrara — Sun, 27 Apr 2008 18:12:13 GMT

I think this would be a great article. Kyle—it sounds like you’re planning to write it with the assumption that the reader is a complete novice on the subject. I think that’s a sound assumption.

I’m terribly uninformed on sales processes, yet I realize that they have very far-reaching implications for a project. I certainly wouldn’t know what alternatives to an RFP would be available. So having something like this to brush up on the more businessy side of design would, in my mind, be tremendously helpful.

John Ferrara — Wed, 04 Jun 2008 15:00:34 GMT

Lukic’s book seems like it’s geared very much toward aesthetics; his concepts strike me as gleefully nonutilitarian. I’d be very interested to know if there’s a functional angle to the nonobjects idea that could inspire design for use. Intangibles are unquestionably a part of a product’s experience, but can they be a part of its purpose too?

John Ferrara — Fri, 13 Jun 2008 16:52:54 GMT

I think this is badly needed. Pragmatism demands that these documents be written with the assumption of some minimum level of “wireframe literacy” on the part of the reader. Little surprise that’s often absent among many of the key approvers. In the past I’ve provided a one-page primer on wireframes explaining their purpose and notation conventions, but this sounds more like an interactive guide. Very intriguing.

Aaron, when you say that it’ll include code, do you mean that the wireframes will be presented as scripted PDF’s?