Community Articles

Destigmatizing Multimedia: Exploiting Human Attention Mechanisms to Present Complex Data

Mike Johns, Product Research Engineer, NetQoS Inc.

Since I first demonstrated Netcosm to my peers at NetQoS as a bunch of yellow blocks flying between purple blocks, the reaction has surprised me. Within a few days, it felt like Halloween at my office with people stopping by for a look. Two months later, thanks to our Marketing and Public Relations crews, we offered a glimpse of Netcosm to the Internet with results beyond what we hoped. It has been difficult to pin down exactly where this attention comes from. What is it about flaming servers that provokes such a response? Aside from sheer novelty, the answer, I believe, is that Netcosm exemplifies an approach to information delivery that the brain likes.

In the last 20 years, the amount of information available to us has exploded. With this wealth of information comes a new set of problems. With so much information within arm's reach, how do we choose where to spend our attention? There are two approaches to answering this question. Human attention is a bandwidth-limited resource and network engineers are familiar with solutions to bandwidth-oriented problems: send less information or find more bandwidth.

Information usefulness is limited by how much we can cram through the senses into someone's brain and the relevance of what gets through. Information delivered over the Internet has had considerable attention paid to improving relevance via search engines and the like. The better the search engine, the less brain bandwidth you must spend finding what you want.

Filtering has limitations. Sometimes we must sift through noise to find the signal; therefore, information providers look for simpler ways to present information. Opening bigger pipes into the brain does not make sense because we cannot upgrade our fundamental equipment. The attention paid to increasing the information volume the brain can process has manifested as the art and science of streamlining user interfaces, but today's business applications tend not to fully use our natural equipment. Everything tends to get forced through the channels that process text and images. While these are important methods for delivering information, interpreting graphs is a somewhat difficult task for the brain. Perhaps we can process more information in less time by expanding our repertoire. Instead of reducing the amount of data we send via machine preprocessing or a perfected user interface, we can rely on the brain's abilities to filter information when it is delivered in certain ways; specifically, as animation and sound. Humans are exceptionally good at detecting slight anomalies in movement and sound patterns. This innate skill helped us eat when we had to hunt our own food and now it might help us eat when our food is somehow tied to processing data streams via our jobs.

It is possible and indeed likely that making use of this expanded bandwidth to the brain allows a disproportionate amount of non-information into the stream. This is generally the problem with multimedia presentations and the reason why many of us recoil at the thought of anything animated. Additional demands are put on our attention without additional payoff. Historical misuses of such methods should not preclude us from continuing to try to find better answers. Awareness of such pitfalls can help us avoid them.

By expanding our methods of information delivery to those the brain is readily equipped to process and flooding those channels with legitimate information, we can minimize the amount of work the brain has to do. It can take thousands of graphs, most of which would show nothing noteworthy, to convey the information shown in 15 minutes of Netcosm animation. Instead of deciding how to tell interesting from uninteresting graphs, we can present everything and defer to the natural abilities of the user's attention mechanisms. Indeed, much of a Netcosm visualization is also showing nothing noteworthy, but it is easier to focus on what is most interesting in such an animation than it is to sort through thousands of graphs or automate such work. When something attracts attention, the full data detail can then be examined in whatever form the user prefers. By presenting metrics related to speed as actual rates of movement and metrics related to volume by scaling onscreen entities, we limit the need for creative interpretation on the designer's end as well as the user's. The result is a representation that greatly simplifies the work needed to identify and interpret the important parts of a dataset.

I wish I could say that I began work on Netcosm with these principles in mind. As it turns out, they only became apparent after the fact. Only after realizing that I was becoming acutely aware of our internal network performance as a result of developing Netcosm did I begin to believe in its value as a serious tool.

The visual metaphor was strongly suggested by the data in question: end-to-end timing measurements. Each record in the table represents a 5-minute aggregation of the response times experienced by packets of a particular application type from a client region to a server. I started by drawing the clients and then the servers and animating objects moving between them at a speed influenced by the network round trip time. Upon reaching the server, the object waits according to the server response time indicated by the data and eventually returns to the client region as a response. It was difficult to imagine doing this any other way; thus, it served as a solid starting point.

Eventually, I introduced the idea of scaling the size of the request according to the number of bytes measured to and from the server. This allows viewers to note periods or locations of particularly heavy traffic. Initially, there was no way of telling what application issued a particular request. There were two obvious ways to remedy the former: vary color by application or vary shape. After some experimentation, the advantages of shape variance became apparent. Users have strong associations between applications and certain images thanks to their icons. Flying folders are instantly recognized as file transfers, cylinders as database traffic, envelopes as email, and so forth. These icons eliminated the need for users to learn arbitrary color schemes.

With potentially thousands of client regions issuing requests, it was difficult to visually trace slow-moving requests to their sources. I remedied this by shading client regions according to the round-trip times experienced by recently issued requests. Inactive client regions fade into the background and active regionsare colored somewhere between bright green and bright red according to the worst round-trip times experienced by issued requests within a time period.

The large number of requests traveling to and from servers made it difficult to judge a server’s response time at a glance. The causes of server response time issues are referred to in dramatic terms. Servers are said to melt down, blow up, fall over, and so forth. This perception allowed me to liven up the visualization with pyrotechnics. Veterans of Grand Theft Auto recall the approach used there to indicate car damage: smoke means stop running into things and fire means get out of the car immediately no matter how fast you might be moving. Inspired by this method of getting the point across, I introduced a similar effect for servers and this became one of the memorable aspects of the visualization. When it comes to drawing attention, fire almost always does the trick.

Excessive retransmissions are a major indicator of network problems. The newly introduced capability to blow things up provided a natural way of expressing them. Packets are retransmitted when something happens to them along the way such that they do not arrive or arrive in a damaged state. This led to the visualization of retransmissions as exploding en-route. For sessions refused by the server, the server shoots them in the air where they explode. With accompanying sound effects, this dramatically makes users aware of problem,s even when they are not looking at the screen.

A few days of this auditory barrage led me to believe that sound cues can be immensely valuable in an application such as this. This was not a unique observation. I wanted to try something like it since hearing about other projects that produce auditory representations of network data. I introduced an effect where the average network round trip time of all visible requests influences the pitch of a constant hum emitted by the visualization. When the network slows down, the sound gets deeper. When the problem clears up, the sound returns to normal. Sound cues users into issues that occur along a continuum instead of in discrete transitions. Small variations in pitch are readily ignored or not perceived where larger ones claim attention.

With all of the above in place, a common remaining feature request was to introduce a way of organizing onscreen entities according to their real-world geography. The capability to drag client regions and servers around on a map was introduced as a result. This allows the eye to travel among related groups and detect problems within a particular group or across them. Further, this enhances the element of familiarity to viewers and allows them to quickly get their bearings within the visualization.

In the end, what mattered most was that I actually enjoyed watching it. Like an aquarium, the continuous motion was soothing. In fact, I briefly entertained the idea of calling it Netquarium and introducing an underwater theme. In the end, fire and explosions proved too compelling and made little sense underwater. The idea might resurface, no pun intended.

The aesthetic appeal made me more likely to run Netcosm when something interesting happened. I could see what was going on while sitting idly in my chair, even when primarily engaged in something else. When an atypical event began to unfold, the deviation from the normal pattern was immediately apparent. Whether I was watching attentively, talking to someone, or ALT+TABbed in another application, the visual and auditory cues immediately grabbed my attention and then let it go when things went back to normal, which, as someone researching methods of getting the important parts of large datasets examined and understood, was exactly what I hoped would happen.

View a video sample of Netcosm.

Loading...