Boxes And Arrows : The Design Behind the Design

Hi Ross:

As I mentioned before, great article. Not sure about the value of the XP Bliss wallpaper but heck, I will choose to ignore that reference! ;)

I suggest a visit to an oldie but still interesting site reference: http://isseymiyake.com (click on autumn-winter 2002-2003). While the site is totally not usable for corporate structures, it is interesting for how they have used the glowing circle to mean navigation quite consistently…the audio cues are also consistent and set up expectations from the first screen where you choose men/women.

also interesting is the “map” and how it orients the user to a breadcrumb of where they have already been.

combine “ambient signifiers” with pavlovian conditioning and you could set up a very positive investment-reward scenario for users.

This was cool, and I definitely think it needs more exploration. Specifically, guidelines (and anti-guidelines) so designers can more easily make their signifiers more ambient.

Tufte might offer some additional examples as well. His recommendation is to always use the least possible difference.

I can’t remember which book, but in one (or several) of his books, he has some nice samples where he’s redesigned really aggressive pieces and communicated more effectively by removing entire design elements, muting colors, and lessening the presence of of other elements (like type, lines).

Amazing ideas here. My company always uses custom-scored music and frequently employs ambient background noise to evoke emotional response to an immersive, rich media environment. However, I don’t believe we’ve ever attempted to use sound and scoring as a navigational aid.

I also wonder if there aren’t significant implications for bringing a mixture of emotionally evocative qualities and usability through sound for disabled Web users. A sight-impared user, for example, is essentially what a Tokyo subway rider is if they never look up from their novel.

I appreciate your article…definitely thought-provoking.

I think this is a very important concept. However, if every website tried to accomplish this in a different way, the internet could become quite confusing with hundreds of ambient cues overwhelming users. The beauty about the Tokyo Rail System is that the same principles are carried out universally. For the web, I think consistent ambient cues from one site to another are very important for the success of this idea.

In the HCI community, the sort of ambient audio signals you’re discussing here are generally called “earcons.”

I would not make the mistake of collapsing ambient signals with those that occur at low frequency. Indeed, it may be difficult to learn a “language” of cues invoked at infrequent intervals, and many of the most successful ambient signalling channels are those that are engaged constantly or nearly-so.

Just to pick up on a minor point at the risk of coming across like a neo-Marxist nutjob…

The following comment in the article may seem at first innocuous: “An online publication could feature this month’s articles in high-contrast colors, while previous months are rendered more subtly.” However, I have a big problem with news sites, editorial control, and the idea that what goes for offline news should apply to the online variety. The problem I have can be summed up by the question “Why do I need to be told what is important news, by somebody I do not know, who knows nothing about me?”

This isn’t so much a UX issue I know, but I find it immoral that somebody should edit online news, let alone design a system that ingrains this editorial action. Offline news I have to accept is constrained by physical space, so I can tolerate that – but not on line. Sub-editing, yes, authoritative comment, fine, but deciding what I need to see, or what is important – no.

Nice article though, since this is an area that needs more attention in the overall design process. BTW the BBC’s “patina” idea was retired a couple of years back for being far too subtle, and in my opinion, pointless.

I would like to thank everyone who read this article, and also the people who have taken time to write comments and feedback.

I’d like to re-iterate that this article is not about using sounds on the web (not that I am always opposed to that idea). It merely uses the Tokyo rail network’s adoption of melodies as an example of ambient signifiers.

I agree that ambient signifiers may become even more effective if they form and adopt conventions. I suspect these patterns will emerge through further research and maturation.

My use of the word ‘frequency’ was a little too ambiguous. Ambient signifiers tend to be constant throughout the session – essentially occurring only once per signifier (but persisting). High frequency elements such as icons and textual data tend to be physically smaller and independently repeated (with a high occurrence/recurrence).

My examples are merely there to demonstrate possible uses for ambient signifiers. They are not intended to define appropriate adoption or utilisation. One could debate the morality of editorially controlled content, but this is another topic in itself – albeit an interesting one.

I was aware that the BBC’s “patina” was no longer in use. I would be interested to see any research and user testing that was done into its original goal and subsequent results. Personally I think that whilst it may not have been a particularly successful implementation (who knows?) this does not preclude the use of ambient signifiers in general. But clearly more research and evaluation needs to occur.

can’t remember which book, but in one (or several) of his books, he has some nice samples where he’s redesigned really aggressive pieces and communicated more effectively by removing entire design elements, muting colors, and lessening the presence of of other elements (like type, lines).

I think a cleverly designed subway system has a deep impact on society. In a sense, it teaches the public to think logically. If a subway system is confused or disorienting, it can make the whole world feel disorganized. It would be interesting to examine how people apply the patterns of their subway system throughout their lives.

Ross, I’m glad to see your clarification. I’d like to propose that for consistency’s sake we take the next step here, and define ambient interface elements as those that remain peripheral and are not ordinarily focally attended to by the user. (This would both bring our discussion into line with standard usage and allow us to eliminate any consideration of “frequency.”)

That said, I feel that the use of ambient audio-channel indicators remains an interesting and underutilized strategy in interaction design. The MIT Media Lab Tangible Media group’s 1999 ambientROOM is a crucial early elaboration of the idea; I suppose the whirring of Natalie Jeremijenko’s “Live Wire” (also known as “Dangling String”) could be considered a still-earlier example. While innovators like Schulze & Webb continue to explore the possibilities of ambient display in the visual channel, relatively few recent projects have fully exploited the possibilities of ambient sound.

Great article.

To me, the best thing about ambient signifiers is that user’s don’t have to consciously recognize them to accomplish tasks on the site. In other words, since they stay secondary, users who don’t notice the signifiers (consciously or subconsciously) will still be able to go about their business. This independence from functional navigation and wayfinding should be a key qualifier for determining if something is an ambient signifier.

When implemented well, this type of system should not introduce confusion since most users won’t even be aware of it – with the net effect ranging from zero impact to a very positive impact on user experience.

I quite accept that your examples are merely there to demonstrate possible uses, but I worry when I read articles like this that posit a good thing (like ambient signifiers) that don’t then then follow through with solid examples. To take an extreme position: if the Tokyo underground usage is the only good example, then that’s an indication that perhaps the idea is not actually valid. The BBC’s “patina” experiment does little to show whether ambient signifiers on the web are appropriate or indeed even useful. My own opinion is that a web patina is not a good idea, since it would by definition discourage users from looking at content that they may well be interested in (for example, a temporary attractor hook). The fact that it would, over time, increasingly limit their ability to notice other things on the page makes it even worse in my opinion.

However, please don’t see this as strong criticism. I am very grateful to you in writing this article beause it’s given me pause for me to think about this.

After reading your article, I’ve tried very hard to find examples in non-web applications that employ these techniques. Off the top of my head, Firefox is a great example – not only the yellow address bar to signify security, but also the red search box in the bottom when your search item is not found on the page. MacOS X, when searching in the preference pane, desaturates and darkens the window, except for the icons relevant to your query; the icons that have partial matches appear colored with penumbral shadows, and the full matches appear with umbras. But perhaps the best example I’ve seen comes with certain Unix shell customizations that change the user prompt to a different colour if you’re logged in locally or remotely, and darken and lighten the prompt according to the laod of the machine.

I mention this because I think there’s great room for improvement in these areas. Perhaps the MacOS X dock should also change its hue slightly to reflect the machine load. Connectivity to the internet could be indicated by making the shadow of every window that has accessed a remote IP slightly red (Imagine what a great security feature this would be!) How about physical mediums? Library overdue notices should be printed on paper ranging from light pink to red (I receive multiple for the same book, but naturally, I ignore the first few) – similarly, package notices (from USPS, UPS, and Fedex) could use a similar system. Subway systems could use the improvements the Tokyo line has, and more: in New York, the signage leading to the north exit should be a different color than the south.

Much can be done with pigment altering chemistry: Thirty-day Metrocards should be printed on special paper than gets increasingly redder as its expiration aproaches. I’ve seen something similar done in some parking spaces, where either by accident or thoughtfulness, the machine dispensed ticket you stick to your window fades out completely in 24 hours, allowing an inspector to immediately spot a violation. Cups could show the temperature of their contents.

It’s great to see this issue come up again, with such a thoughtful approach (and great discussion). In grad school many moons ago I was part of that auditory interface scene (if there was ever a scene) Adam refers to. Some called it “earcons” while others felt that term wasn’t broad enough and on and on it went.

For my thesis I tried and semi-failed to create a tone language for navigating hypertext structure. This was pre-WWW, and was based on some naive assumptions about how people could or could not learn to move around virtual space. The research was really rudimentary but there were some intruiging things that people COULD learn to do. You can see a PDF of a paper I haven’t looked at for a long time at http://www.icad.org/websiteV2.0/Conferences/ICAD94/papers…

This is a really interesting thread—I too would like to see more work done on ambient signifiers. One thing to occurred to me in reading some of the responses is that how ambient signifiers could be used on the Web probably depends at least to some extent on how the particular Web site in question is being used. I wonder if ambient signifiers wouldn’t be more useful on internal organizational websites—where members of the organization are likely to be regular users and may have more of an opportunity to train themselves in responding to ambient cues.

It seems to me that an important issue related to ambient signifiers would be: how to use perceptive cues to create a coherent ground?
I mean: what looks great to me when I use Mac OSX is that the window’s shadow and elements (keys, buttons and colours) are the same for a folder window, an application window and a browser window. On one side we have the nonsense of calling all these components with the same name (windows), which should (and does) confuse the user. On the other hand we have some “ambient signifiers” based on salient perceptual figures that create relations between different areas of the screen. That’s not just estethic, it’s pragmatic: figures are used to act in unknown contexts. To use your example, in a Tokyo train a user who have lost the cognition of which line he is using at the moment, can be helped by a sound at the next station. In the same way, the user who doesn’t understand how to delete a file from a new application he’s using will be helped by a “recycle bin like” icon or he’ll try to use a combination of keys he is used to. I developed these ideas in my thesis on the semiotic of GUI (if you want to know something more read my suggestion: 3722) and I’m glad to see a similar direction in what you wrote.