Boxes and Arrows: Comments by Steve Fleckenstein

Steve Fleckenstein — Wed, 31 Oct 2007 00:44:47 GMT

The title of this article should be “Log files: 99% Bad” because, like Nielsen’s article of similar name this one seems to have been written in 2000. While you focus on log files—which I agree have a lot of limitations—your article completely ignores client side analytics tagging (which addresses many caching issues and which is used by thousands of sites). And while I agree that it is difficult to measure online success via analytics (but that too is possible to some degree with advanced features of some web analytics tools that can track a purchase funnel) I can still get a lot of great info out of web analytics tools that help me improve the site for users. A more balanced view would have been helpful—“log files aren’t all that helpful but client side tagging is a viable workaround.”

Steve Fleckenstein — Thu, 25 Oct 2007 16:29:28 GMT

Here’s an example. I discover from analytics (regardless of method) that a certain detailed content page buried in my site is requested many thousands of times. I’m using the word “request” specifically here to indicate my agreement about the inherent limitation of analytics. I also know that a link to this page does not appear anywhere in the top 3 levels of the site. Even without user testing (which is important, I’m not trying to debate that) would it be reasonable to draw the conclusion that users would be better served if I surfaced this content to a higher level?

Steve Fleckenstein — Fri, 26 Oct 2007 12:49:34 GMT

Cennydd, well stated. Karl, thanks for the response to my earlier comment. I agree with your last statement—we shouldn’t make changes based solely on analytics. I like Cennydd’s statement that analytics provide a good starting off point for future investigation (especially if combined with search log analysis and other information). Bots, caching, etc. all complicate things but again that doesn’t mean I should ignore web analytics completely.

Steve Fleckenstein — Thu, 25 Oct 2007 20:24:54 GMT

P.S. And Karl, my apologies for the inflammatory opening line to the first comment—it deserved the negative rating it received. There’s lots of good analysis in the article and it was irresponsible of me to toss the baby out with the bathwater, which I… er… think I accused you of doing.

Steve Fleckenstein — Tue, 01 Apr 2008 17:57:36 GMT

Daniel, thanks for the article. I’ve found that this approach (combined with other approaches as noted by previous commenters) works really well. I’ve also found that working with surrogates can sometimes help engage people throughout your company, turning them into allies that you can approach in the future for assistance.

One additional thing to keep in mind when using surrogates: it’s helpful to match surrogates to the users they serve. That way you get a usable amount of information on each user group. For example, in your Quebec city example you interviewed call center reps to get info on consumers but you probably also spoke to people in the permits area to get data on the needs of building contractors