Kenneth Walters — Fri, 07 Jan 2011 19:41:23 GMT

I agree with your premise, but this article does leave me somewhat frustrated.

In the blog style examples (TMZ and News Daily Pulse) and the non-blog example of Top Tax Tips, the items that you are measuring are part of the main flow of content. This is what is taking the user down the page and I would expect that, given strong content, these items would do reasonable well.

I’m disappointed that the one non-main content flow example that is used, AOL Money & Finance, is the one that doesn’t provide any form of quantitative measurement.

“Users find and use the modules for recent quotes and their personalized portfolios even when these modules are placed well beneath the 1024×768 fold.”

Does this mean 2 users or 99% of visitors?

Even where numbers are used, they are provided as raw numbers without context or any basis for comparison or evaluation.

The Top Tax Tips example states 200,000 to 400,000 page views. Is this good? Is this 5% of the visitors or 95%? What would these numbers be if the content were presented without the need for scrolling?

I’m not disagreeing with the principles, only the lack of sophistication in the analysis. When people start throwing around specific measurements as statistical evidence, that evidence needs to be sound and fully disclosed. Otherwise it’s just as bad or worse than using words like “lots” or “many” or “most”. They could be worse because people will look at the numbers and accept them as unquestionable truths without real or complete research.

Boxes and Arrows: Comments by Kenneth Walters