Boxes and Arrows: Comments by Matt Queen

Matt Queen — Fri, 26 Jan 2007 14:52:31 GMT

I read this in a section titled “Highest-rated suggestions.” I thought that is was an invitation to write this article. So I clicked on the link “Submit your story now” under “Write for Us.” To my shock, my post appeared in the suggestions section! (Along with yours). After reading a bit about the new review process, I see that the site inadvertently lead me to step on Scott’s toes. Of course you can’t delete posts, so I’ll just request my post be deleted. Feel free to email me to compare notes before your article draft if you’d like. Sorry for the confusion.

Matt Queen
queen@synergia.com

Matt Queen — Fri, 26 Jan 2007 14:52:31 GMT

Sorry, I’d like to rescind on this offer.

Matt

Matt Queen — Fri, 26 Jan 2007 14:52:32 GMT

Thanks Liz

Matt Queen — Wed, 23 May 2007 13:17:03 GMT

Glad I could help with the lingo! There is another distinction though. Suppose you ‘diagram’ 1 user during a session. I would be inclined to call that a flow diagram too! However, suppose you diagram 10 users? Then, user cognitive states like “edit list for X” becomes, “edit” as more users exhibit the same behavior. Then, you get some predictability (with a level of abstraction). More users will do this edit stage! As soon as corroborated predictability sets in … then it’s a model. Intuitively, the phrase “model predictions” is commonly used with that same meaning. BTW, that speech always gets a standing ovation :)

Matt Queen — Fri, 26 Jan 2007 14:52:32 GMT

Of this you speak the truth. And, your comment, ”... believing they will get a ‘free design’ and ‘save money’ having the programmers sort of ‘design-while-coding’.”—sounds like the voice of experience :)

That is a case study I’d like to read.

Matt Queen — Fri, 26 Jan 2007 14:52:37 GMT

Hi Amanda. One way you can coerce your concerns into a research question (fit to write and article about) would be to express it as the following question: “What do users need but seldom ask for—and why?”

Your answers might relate to how user interviews are structured, how data was collected, how stake holder interpretations developed, how much stake holders actually care about user data, etc. In short, this research question is fairly open and has many possible directions. The answers would be interesting to read about. If you could investigate answers to the research question and present the results of your investigation in a meaningful case study, this research will fly!

I’d be happy to be a good colleague and trade ideas about this subject with you in prep for your investigation—send me email.

mqn at u.washington.edu

Matt Queen — Tue, 11 Jan 2011 03:39:37 GMT

Your concerns seem more closely related to semiotics. I am certainly no semiotics expert. Also, I agree with your comments about the usefulness of icons on the web. I completed a draft of this article a few days ago, here is an excerpt that explains a use and setting for this analysis:

“Systems used by first responders in hazardous materials incidents (see MARPLOT for example) rely on icon design to signify entity classification (e.g. small icon of a schoolhouse) and level of critical danger to an entity (e.g. a school icon is painted red on a map). Immediately recognizing danger to a school amongst lumber yards, garbage dumps, and plant nurseries is imperative. Any time slip in the search and discrimination task could delay notification and evacuation of hundreds of children. How then can we diagnose problems with icons that fail in this regard?”

There are a host of web applications that fit this characterization:
http://yosemite.epa.gov/oswer/ceppoweb.nsf/content/ds-noe…

I agree the study of semiotics is important yet the field is not very well worked out. There are some good authors to recommend though:

Knowlton, J.Q. (1966). On the Definition of “Picture.” AV Communication, 14, 157-183.

Or for review of the seminal lit.

Anglin, G., Towers, R., & Levie, H. (1996). Visual message design and learning: The role of static and dynamic illustrations. In D.H. Jonassen (Ed.), Handbook of Research for Educational Communications and Technology. New York: Simon and Schuster Macmillan.

Matt Queen — Fri, 26 Jan 2007 14:52:46 GMT

small-quick-dirty explorations are my favorite. I would love to see this written up with a discussion unpacking these results.

Matt Queen — Fri, 26 Jan 2007 14:52:47 GMT

Also, glad to see Fitt’s law being used for its actual purpose! The law gets tossed around so much.

Matt Queen — Tue, 02 Sep 2008 06:21:35 GMT

I haven’t heard of The Bridge though many similar methods exist in the expert systems lit. Practice mapping, for example, is a much more complex version of interaction modeling that often involves decision analysis techniques in lieu of the cognitive bias warrants I described above. Was there any mention of whether The Bridge enjoyed any success (case studies, anecdotes, etc.)?

Matt Queen — Fri, 26 Jan 2007 14:52:50 GMT

The most well known HCI work involving Fitt’s law was done at Xerox—truthfully, I haven’t heard of any other major studies that would provide benchmarks. Perhaps you could follow from here:
http://en.wikipedia.org/wiki/Fitts‘_law

and find something though.

Matt Queen — Fri, 26 Jan 2007 14:53:03 GMT

I just mentioned in an email to Dustin Hamilton that he hit the nail on the head, distance filtering of icons is an evaluative technique—not a design technique. My knee-jerk reaction of how to coerce this method into a design tool is to create a plug-in for Photoshop or GIMP (preferably) that would allow an auto-updated window of the design space filtered at set distances. That way all design decisions (small and large) could be made in the context of the filtered frequency views. I imagine there is someone out there with enough talent and time to create such a tool. It seems like it would be useful.

Matt Queen — Fri, 26 Jan 2007 14:53:04 GMT

I’m not sure whether designers rely too much on icons though I know they are a popular way to represent actions (cut/paste) and categories (media file vs. word document) – which requires care (ex. “is this icon on the web an action or a category, or both?”).
The necessity of having 20 visually unique and distinguishable icons sounds like quite a design problem … the main problem being, “why is that a necessity?” Suppose we cut this 20 to 15 and partition them out. For example we have a web application that has 3 modes while each mode has a toolbar holding 5 icons. The user only sees 5 icons in the toolbar at time and we will decide that icons appearing in menus are preclassified differently in the mind of the user than icons that persist in the UI. The point isn’t that a user can distinguish between a low frequency representation of an icon in the first toolset and the second. The point is that they can easily distinguish between the currently available tools (then we leave the problems arising from modal issues to another type of analysis). Finally, suppose we would rather not rely on icons (after all they can be expensive) and opt for well thought out labels in our UI. Believe it or not, labels have low frequency components as well! The large letter forms of capitols, risers, and descenders (ex. “X”, “t”, “g”) coupled with the amount of kerning and line space contributes to the recognition of labels. We use the P channel (specifically, “detailed shape”) to read the labels yet rely on low frequency components to aide the identification of a word. Some reading research suggests that we actually don’t “read” all the letters in a word. Rather, we recognize a word by its large distinguishing visual features. We could argue that large visually distinguishing features are low frequency components. Does this make sense (perhaps improve the usefulness of this technique for you at all?).

Matt Queen — Fri, 26 Jan 2007 14:53:04 GMT

Yes I agree Rex. The current state of knowledge in neuroscience and cognition supports the mix of top down (recognizing/identifying) and bottom up (sensing/describing). Following the discussion into icons and signs, you could look up the works of Joan Peeck, “The Role of Illustrations in Processing and Remembering Illustrated Text” and Robert B. Kozma’s “Learning With Media.” I think those are good references for looking at icons and illustrations. They aren’t neuroscience so you won’t find any discussion of retinal architecture but they are chalk full of wisdom about how users learn and absorb information from media. As for research specific to icons and spatial frequency – as far as I know – you just read it! If others have found articles of this nature post them: I’d be interested.

Matt Queen — Tue, 11 Jan 2011 03:39:35 GMT

Well – I have some response to perform here! Economically, and in order of appearance:

Andrews:
Good point about text. There is less variation and more standards for text patterns resulting in easier pattern matching from peripheral vision images—so long as you read the language and have a healthy vocabulary, and perhaps are familiar with domain specific terms, font choices are typical or familiar, etc. However, 25 icons in the browser is a lot. I use the back, forward, refresh, stop, “kill the tab”—in firefox, minimize, maximize, close the window, and I suppose you could make an argument for the firefox window logo. While we’re at it I suppose you could also argue the fringe use of icons in the bookmarks menu (folder vs. doc), but the window logo and bookmarks icons are marginal arguments and could be classed outside of browsing behaviors for most of use time (I would argue based on my behaviors anyhow). That’s a thorough 11 and while there may be more (in the context of web pages and such) you could also argue that is different software. Though I wouldn’t try to instruct on the appropriate quantity of icons for general cases, I feel most applications employ a reasonable number yet allow them to persist in inappropriate ways. A good technique (by counter example) is: for icons representing local actions for, say, a list item—have the icons appear when the user places the mouse in the vicinity of the list item. It doesn’t persist, serves as a good reinforcement (your mouse has to be here when you do this), provides feedback (i.e. “you mean this list item, right?”), and classes the local options in a different cognitive mode for the user. I’ve seen that work well in apps before.

Zapata:
So long as you maintain good form, your squint test is still alright :)—Good points about applicability, yet try hard not to use this technique for evil (I’m envisioning all sorts of subliminal brain washing attempts involving low spatial frequency).

Laird:
Categorizing by function starts to get into looking at other components of the icon rather than sticking strictly to perceptual acuity. This is a great idea because it would start to draw relationships between the various components of an icon. I’ll say more about this later I suspect as I make my way down the list here.

Roast:
I checked out Barnard and May (I assume you meant, “Modelling User Performance in Visually Based Interactions”). I got stuck reading several of those pubs—great stuff. The issues they were teasing out were far more in the territory of what Laird was proposing above. There is a bit more resilience in investigating those issues because of the extra muck involved with higher cognitive processing. The analysis I’ve proposed here mixes some of that muck in the breakfast cereal though doesn’t it? A user has to compare a perceived pattern (low level) to one in stored memory (pretty high level) in order to respond in a measurable fashion. That isn’t the largest problem here though – I’ll say more about this later.

Parks:
I checked out your site and it looks like you have some interesting work going on there! I also looked at the school/dept. pages and about Ottawa and your local area too. I just have to visit! I’m told Paesani’s Caffe on Preston is a must as well.

Chee:
I also read Mullet and Sano, great work—if you liked that you would also enjoy the early works of Cleveland and McGill, and for some cool background reading (and weird stories) a book called “wet mind” by Kosslyn). I don’t have any case studies involving this technique or similar methods. When I get a chance I’ll look around at some systems and make some notes. So I’ll get back to you and post what I’ve got.

Houx:
First, thanks for taking this on! Second, thanks for emailing me and alerting my attention to the mounting list of comments here! Finally, the R code simply uses the fast fourier transform to convert the image to frequency space (it does this in two dimension for the images sake though) then cuts the high frequencies out using our angle argument—then inverse fourier’s the result to return the signal back to image space. You may have already figured this out but if you just type “lowpass” on the command line in R it gives you the code used. The image has to be normalized first, but the R code will guide you to the appropriate source code file that actually does the pass filtering and you can also chase down the fast fourier implementation from there too. Email me with any progress, questions, and what not if you’ve got ‘em.

Darren:
That sounds like you got it. The M pathway is only sensitive to “gross” form and while the P pathway is sensitive to sharp details, owing to the difference in size of the receptor parts (anatomy). And the image you linked is, in fact, from Schyns and Olivia.

Dagi:
The 10 users weren’t supposed to be a sample size – they were supposed to be an “example size” (Emma Rose really knows how to sell that line). This is the way I look at it. Icons have several components: perceptual, semantic, contextual, etc. Each of these components can be engineered and optimized. The optimization will almost always involve relationships between components, Ex. “this graphic results in less confusion [semantic] on a button vs. on a map [context]” (of course, you run into the problem of trying to measure “confusion”, but that is a different story). With a research endeavor it is often the same process, and so too is it here. To really make this a useful experiment design, as I have stated before, the problem here isn’t stats and sample sizes and populations, etc. The real problem is the inability to strike a good robust relationship between the engineered difference of low frequency patterns and response time standard deviations. Ex. The icons used in group A were different in low spatial frequency components which lead to a conformational drop in overall response time and deviation of times among group A user trials. The glaring problem with that statement is that “different in low spatial frequency” part. How different are we talking here? What’s more, how does “really” different relate to a standard deviation of X? Without robust relationships here we end up ascribing causality when the evidence only suggests correlation (the responses were just faster, it didn’t actually have to do with frequency anything though – in fact it could have just been ordering effects, like “more practice.”). To use an analogy, in order to put low spatial frequency improvements to the test we might construct an obstacle course for our low frequency unique bicycle (stay with me here). If our bike can make it through the course in reasonable time (by say an alpha level of X) then our technique will be provable. We need to realize that it’s the robust relationships between the bike parts that will prove this bikes worth. In other words, we will not be able to negotiate the necessary turns if our bike has handle bars made out of cheese. In fact, no amount of doping is going to help us—it’s the cheese handle bars. So as I stated before, it’s the quantification of pattern difference that is the problem—that is the cheese handlebars.
You make some good points about the current use of information graphics in software. You are very right. However, we could rephrase your statement, “maybe we’re asking too much of icons” to, “maybe we’re asking too much of traffic signs.” I generally hold that even if something like an icon is meant as nothing more than a trendy marketing mechanism, it still must conform to the same tests of effectiveness as those on our HAZMAT maps. The icons that must maintain an extreme economy and effectiveness of communication to users who are frantic and desperately trying to hold composure while making the best decisions with the information available. As I’m packing up the proverbial soap box, I’ll admit that some find this attitude to be a bit drastic and opt for, “these are good until the users show us they aren’t.” Of course, how do you know if you don’t measure? And, how do you know you are measuring what you think you are measuring (internal validity of analysis).

I guess that was Dagi and Houx part 2.