Article Idea:

Strategies for Improving Enterprise Search

suggested by John Ferrara on 2007/06/23

There are fairly few B&A articles on improving the findability of information from in-site search. In part, this may be because a search engine is seen as the domain of developers or mathematicians. In fact, it very much needs the skills of an information architect, and the popularity of search proves it’s a problem we can’t ignore.

This article will outline four strategies for improving the quality of results returned by a website search engine, addressing:


  • Content

  • Indexing

  • Query formulation

  • Results display


Topics discussed will progress from structural markup and metadata through ontologies, syntax conventions, query expansion, server logs, and best bets.

While the article will be densely informative, my objective is to make it as light and entertaining as an article covering things like “syntax conventions” can possibly be. If I can’t, then it will probably end up being unintentionally humorous – which, by the strictest definition, still counts as entertainment.

Chris Cavallucci's avatar

Chris Cavallucci

1 Reputation points

Posted 2007/06/24 @ 06:07AM with

I would be interested in your coverage of the query formulation and expansion topics. Google Suggest offers a newer interaction style when it comes to the user completing forms. Are you going to discuss this feedback-on-the-fly approach in designing the interaction?

Dave Cooksey's avatar

Dave Cooksey

2 Reputation points

Posted 2007/06/24 @ 06:41AM with

John – I would be interested in this topic not from the perspective of someone who is in charge of in-site search but as someone who would like to influence its development. In my world, control of search falls in the laps of the product managers, coders, and business folk (strange, but true). Information that would help me frame an argument from the perspective of UX would be of great value to me. – Thanks!

John Ferrara's avatar

John Ferrara

85 Reputation points

Posted 2007/06/25 @ 04:32AM with

Responses to 2 comments above:

Chris—Glad you brought up Google Suggest, I’ve been debating whether I should include discussion of it. On the one hand, I believe that it is a borderline revolutionary assistive tool, and very much a part of the future of search. On the other hand, I intended this article to be broadly pragmatic—and suggest is a function that may be beyond the means of +90% of the readership. I actually had an overview of it in an initial draft of the article, but pulled it in a subsequent version reasoning that I could easily write a full article on suggest functions alone. Any inclusion of the topic in this article would have to be unfortunately cursory because it shares space with so many other subjects, but if there’s interest I’m more than happy to work it back in.

Dave—The perspective of the article would, I believe, be completely in line with your expectations. I think it’s uncommon that UX folks are ever the ones in charge of search, and the article is meant to provide a framework for discussing and influencing its development. I think it arose from the same sorts of concerns you’re citing.

Great comments, thanks so much!

John

John Ferrara's avatar

John Ferrara

85 Reputation points

Posted 2007/06/25 @ 04:38AM with

FYI—for some reason, a portion of my comment above is rendering with strikeout formatting; I think I accidentally used a platform-specific syntax. Please ignore the format, the struck text is meant to be read.

John

Mike  Heck's avatar

Mike Heck

1 Reputation points

Posted 2007/06/25 @ 16:43PM with

Hi John,

I think this is an excellent idea. I know there’s been a lot of talk in academia on this topic—but I believe you can turn it into something very relevant and readable. For what it’s worth, some of the lesser-know players that I like in this space are Siderean (http://www.siderean.com/), exalead (http://www.exalead.com/), and still one of my favorites, Vivísimo (http://vivisimo.com/).

Mike

chris daly's avatar

chris daly

1 Reputation points

Posted 2007/06/25 @ 17:37PM with

Good thinking John,

I’d be excited to read about the impact of results display as they relate to the nature of the search itself. Clearly that has connection to the other three components of your research. Specifically, how can the results be displayed to mutually achieve both user-first goals like “what am I looking for” as well as site-first goals like “this content is close to what you want and makes us more money.” Perhaps that is a split best addressed when considering more transactional sites vs. mroe content driven sites.

Thanks!

Ed Mooney's avatar

Ed Mooney

1 Reputation points

Posted 2007/06/25 @ 18:50PM with

John, I like this idea and think it has strong merit. I’m sure many can attest to the situation of being in a rush and not seeing what we need, after a quick skim, we’re heading for the search box. I would particularly be interested in a discussion of the rendering of structured data, to make it more usable. For instance, vCard, microformats, RSS, and other XML formats. In regards to querying, specialized focusing terms such as Google’s “site:” or “link:” prefixes would be interesting, especially the logic of using terms that are easy to remember, and intuitive. Content & indexing would be nice to see in a different light since, as you note, their basis is in mathematics (Cartesian joins, unions, etc.). For your research, you may what to take a peak at the Apache Nutch project, there philosophy is to have a “transparent” search results rankings, hence their results show why the engine ranked the page as it did.
—- Ed

Matt Gregg's avatar

Matt Gregg

1 Reputation points

Posted 2007/06/26 @ 08:39AM with

Hey John,
This might be a little bit of a tangent topic, but it might be interesting to address how the strategies you plan to outline also mingle with external search engines. Depending on the site, a lot of traffic could be coming from external vs. internal search. It might be nice to see how the strategies complement, inhibit or enhance external search. Maybe this would be a part 2 :-)

Amber DeRosa's avatar

Amber DeRosa

1 Reputation points

Posted 2007/06/26 @ 11:43AM with

Hello John,

I would like to hear more about how to improve search for internal corporate sites and content. People are accustomed to information at the tip of their fingers at home. However, within the firewalls of an internal company it has become difficult to find information. Most people ask “experts” to answer their questions. If the information was easily found via search, it would increase efficiency for both the requester of the information and the “expert” who provides information. The “expert” will be able to focus on their daily tasks and the requestor will receive information in a quick and automated fashion.

John Ferrara's avatar

John Ferrara

85 Reputation points

Posted 2007/06/26 @ 19:07PM with

Thank you everyone for a ton of great comments. I’m going to respond to each of the 5 above in order.

Mike,
I completely agree, there’s so much great thinking among enterprise search vendors these days, and their products have really amazing capabilities. Customers are demanding better tools as the volume of unstructured data is exploding within their organizations, and I think that in some cases they expect the technology itself will solve all of their problems right out of the box. But I believe that the usefulness of the product varies with how it’s implemented, maintained, and used.

Chris,
Yeah, that’s a really important question: how the results display should be influenced by the characteristics of the query. I think that the conventional list of algorithmically determined results is just a starting point, and there’s a lot of value in shaping the results to the user’s needs. I’m planning to cover some of this in the article.

Ed,
As it happens, I’m currently doing some work with indexing of microformats and repurposing them for results display, as well as custom search operators. But this stuff definitely falls at the “way advanced” end of the design spectrum, and easily merits its own article. In a general article like this, the available space is going to be pretty tight. If there’s enough interest out there in these subjects, I might submit a separate proposal.

Matt,
Many of the strategies for improving content internally would indeed also benefit external search rankings. Of course indexing, query formulation, and results display would not, and there’s much more to external search engine optimization.

Amber,
Totally. I think that, in general, the quality of results from enterprise search engines compares very unfavorably to that of external engines. Of course it should be just the opposite, since organizations have the opportunity to finely tune and customize their searches to both the constraints of the data and the needs of their users. The problem is that most simply don’t.

Thanks again, this is all extremely helpful!

John

Kellie Catanzaro's avatar

Kellie Catanzaro

1 Reputation points

Posted 2007/07/02 @ 12:01PM with

I don’t have much to add beyond what’s already been stated, but I’m definitely interested in reading this article!

Laurie Snyder's avatar

Laurie Snyder

0 Reputation points

Posted 2007/07/31 @ 11:26AM with

I think too often, search is merely used as a crutch for a poor navigation system or bloated homepage overloaded with too much information. I would be interested in seeing other additional features that a really good search could provide like additional ways of slicing data or accessing it through other formats.

Search is one of those rarely thought about, but important features that nobody ever wants to spend any time or money on. I think it would be great to provide some guidelines that companies could use as a starting point to improving their site’s search functionality.

Register or Login to comment