Comments on The Evolving Homepage: The Growth of Three Booksellers

James — Fri, 26 Jan 2007 14:49:51 GMT

“Given this environment, I took a different approach: we know something about their return (they have very high figures for revenue and unique visitors), so what characteristics of their design could be contributing (or at least not detracting) from that return?”

There was a similar statement in the article, and it essentially begs the question. The initial proposition should be, Does site design increase revenue for Amazon?

Amazon was among the first big names in on-line shopping. I suspect people became accustomed to Amazon’s layout, and over time it became a de facto standard. Other sites copied it, not because it was the best per se, but because they knew visitors would find it familiar.

jEf — Fri, 26 Jan 2007 14:49:51 GMT

[Disclosure: I was part of the team that developed Borders’ online presence.]

One interesting and relevant tidbit is that Borders’ primary objective for its web presence was to maintain its brand and identity not necessarily to sell more product online. Borders was interested in using the Web and Internet technology to maintain effective bricks-and-mortar operations and strong relationships with an already large and dedicated customer base with the expectation that they’d continue to choose Borders for its unique brand and keep visiting Borders stores. (Whether this is valid or even worked is another story…) Borders never intended to compete for online traffic or sales; in fact, many financial analyses pointed to disaster no matter what the online sales volume was because the margins were so low (or even sub-zero). If anything, Borders had improved ROI through their retail convergence strategy, which used the same Internet-enabled systems to improve retail operations technology infrastructure and enable the heavily-used in store kiosks, clerk stations, and inventory systems.

Since the Borders brand is quite wordy, enriching, quirky—always offering a particular point of view—and relatively “sophisticated,” the site design intentionally tried to offer an enriching and personable experience. It may or may not be true that this approach doesn’t lead to more sales, but more sales wasn’t the goal. The Borders store retail experiences are quite differentiated as well. Borders stores are typically enriched environments that encourage customers to sit down in a cozy chair or at the cafe or at a music kiosk and spend some quality time sampling the merchandise.

So how does all this bear on Victor’s analysis? For starters, I think it’s difficult to compare any form or measure of ROI separate from the bigger picture of what a firm has to gain or lose from an online presence. It’s impossible to compare sites without first establising that they have common measures of success. Second, I’m disturbed by the lack of any mention of audience/user needs and expectations in the evaluation. It’s pretty clear that Borders was going after a different audience and thus designed differently. Third, the prominent role brand plays in both e-tail and retail should be addressed as a fundamental way sellers differentiate themselves, create a following, enable a virtual or phyical experience, and ultimately create sales…

This is a great topic and Victor has done a great job getting us to think hard about the main questions. I’m looking forward to seeing the next piece.

Victor Lombardi — Fri, 26 Jan 2007 14:49:51 GMT

RE>How much more information does one need to implement their own avenue of approach to this hypothesis?

That depends on your level of confidence in this analysis. I’m confident enough that it will influence my design, but next I want to further test these issues using other techniques, usability testing for example.

RE>I think you could’ve written the article in one paragraph, placing the research results as footnotes and simply writing the summary in bullets.

I’d love to see your version of the Gettysburg Address ;-) Usability and experience ain’t the same thing.

Will Garner — Fri, 26 Jan 2007 14:49:51 GMT

Dear Victor,

Thank you for a most informative article about the differences in web-site design among three top-tier competitors. I was impressed with your simple, elegant approach the the question, “What makes a web site successful?” If I’m off the mark on your hypothesis, forgive me.

The piece points out several key issues that are important to all selling, offline and online: 1) Good, easy to understand and comprehend images and short links are better than text. Show, don’t tell! 2) Have minimal text on the page; 3) Put most important items, links, etc. above the fold, i.e. the topmost space on a web site that is most likely to fit on the average browser window, on a typical computer; 4) The simpler it is for a viewer to learn the site, the more apt they are to buy from it and return to it; 5) Use several columns for lots of info on a page, which is analogous to aisles in a store, rather than having the page extend below the fold.

How much more information does one need to implement their own avenue of approach to this hypothesis? I think you could’ve written the article in one paragraph, placing the research results as footnotes and simply writing the summary in bullets.

If people do not entiende the importance of this piece, then they weren’t very good bullet-catchers. Our design team is continually praised for simple, elegant web sites that use high-end graphics and images. We also hear from clients and visitors who thank us profusely for applying the age-old maxim: Keep It Simple!

Picante!

Will Garner
Head Bullet Catcher

Lee Fleming — Fri, 26 Jan 2007 14:49:51 GMT

Victor, Mark Hurst of Creative Good does (or maybe did – I haven’t heard anything from his group in quite some time now) usability/site design/metrics research. It would be interesting to apply your analysis to the sites Creative Good determines are best-practice ecommerce sites. For example, the calculation on above-the-fold link density on sites with a such-and-such ROI increase after redesign, etc. (I wish I had some time to spare – I’d start looking through 10Ks to see if I could isolate some of the numbers needed for a full analysis of site redesigns.) And it would be interesting to do this with different types of products.

Also, we know very little about Amazon’s returns based on revenue and traffic numbers because we don’t know how much that traffic costs, or how much the conversion from visitor to customer costs. Don’t forget, until the last quarter, Amazon’s costs were a lot higher than its revenue.

Based on your spreadsheet, the only apparently significant differences between Amazon & BN are that Amazon has 60% more links and, since there are more links, there are more of them above the fold. Based on your spreadsheet, the only conclusion I could begin to draw is “if you want to sell more stuff, put more links on the home page.” And I would reserve judgment on that until I saw the statistics and accounted for things like maybe buyers of pots and pans or electric screwdrivers like links, but book buyers hate links.

You sure did open up an interesting line of thought with this article! That’s why I said it was fascinating. I think it is a good starting point to maybe come up with an ideal set of metrics on ecommerce site design – one that marries the economics with the actual nuts and bolts (ie. based on these metrics, if you want to achieve this, then do that … )

christina — Fri, 26 Jan 2007 14:49:50 GMT

Hey folks: Please, please please go to the feedback form in the about section to dispense advice on how to improve B&A.. we’ve already implemented a LOT of your comments and the site is changing hourly. ( http://www.boxesandarrows.com/about/contactus.php )

Please keep the comments to comment on the author’s words. Thanks you, and thanks for making B&A better!

Victor Lombardi — Fri, 26 Jan 2007 14:49:50 GMT

In response to Lee…while I’d love to actually measure the ROI of these sites, it’d be difficult to get the requisite cost information out of these companies given the highly competitive state of the industry.

Given this environment, I took a different approach: we know something about their return (they have very high figures for revenue and unique visitors), so what characteristics of their design could be contributing (or at least not detracting) from that return?

Essentially Amazon has a very successful site, and I wanted to try and quantify what was different about their design. Before doing this analysis I didn’t really know. Now I have some definite indications.

The analogy to a physical store layout was my subjective analysis. After comparing all this data for weeks it was an intuitive conclusion. Again, it’s not fact, it’s an idea I think is interesting and worth more investigation on a perceptual/cognitive level.

The article is not bulletproof, we could comment to infinity on its shortcomings. If you find something you find interesting and that makes you think a little differently next time you lay out a page for an e-commerce site, I consider it a success.

Thanks for reading :)

Benjamin Bennett — Fri, 26 Jan 2007 14:49:50 GMT

I was actually a little bit disappointed by the results. It seems like Victor did some great leg work in tracking the changes, however I was hoping for some solid conclusions to discuss. Perhaps someone will be willing to put a stake in the ground and state what is “good” and what is “bad” in this oh-so popular genre of commerce sites?

That being said I did appreciated Victor’s analogy of physical store design (although I can hardly say it is anything new to think about.) In any case I found it interesting that he compared the above-the-fold home page design to experience of walking into the store. I learned in marketing class that studies have shown that the first 10 feet while walking into a store are essentially dead space, because people are primarily concerned with getting into the store first. They eventually start to look around, but only after those first 10 feet or so.

If that is true and the analogy between virtual and physical stores hold true, then it would 1) explain the extensive content below-the-fold and 2) demand that the secondary and tertiary pages be analyzed in conjunction with the home page. Both of those options are places for users to “walk” to once they are “in” the store.

Do you as a _____ designer/architect consider and/or agree with “the first 10 feet are for getting into the store” theory?

Lee Fleming — Fri, 26 Jan 2007 14:49:50 GMT

While this was a fascinating article, I don’t quite see the point. The analysis didn’t tell me if any of the three bookstore redesigns resulted in an increased return on investment, increased sales, increased click-through, fewer abandoned shopping carts – all things I would want to know when critiquing the design of an ecommerce site.

Is a design “resembling a physical store layout” really the most effective? Is this even true (that the design resembles a physical store layout, I mean)? Frankly, I don’t see it.

This conclusion: “Like physical stores, those designs should only change gradually to keep visitors buying.” may or may not be true, but it’s not based on this analysis. Also, most physical stores do not retain the same layout consistently – something that seems to drive customers crazy (especially in a grocery store).

Counting pixels and calculating link density doesn’t answer any questions about the impact of a design on sales (assuming a basic usable design to begin with). You HAVE to look at financial measures and usage statistics. For all we know, even though Amazon might generate more sales, Barnes and Noble might have a more profitable design. When the more profitable design is determined, THEN the pixel counting/link density calculation should come into play.