The debate of icon / text and icon + text label always fascinated me. Gmail started out with an all text interface and very few icons, eventually it changed to the polar opposite and went on mostly icons with very few labels. But which is better? And why?
Novice users are not as familiar as we are with certain conventional icons, making an icon-only interface potentially harder for them to use. On the other hand, to the tech-savvy users, having text labels everywhere may be perceived as cluttered and unnecessary. So how do we make everyone happy?
After some googling on the subject I found a fascinatingly detailed study on the usability of icons only, text only and icons + text. You can read it here if you want to know every detail about it, but below are some of my highlights and what I learned from it.
QUICK INTRO TO ICONS
Icons have been used on a limited basis since the early days of computer graphics. The popularization of iconic representation in the interface dates to the work of David Canfy eld Smith and his colleagues who developed the interface of the Xerox Star workstation (Smith et al. 1982; Johnson et al. 1989). Icons are used in the interface because they are presumed to facilitate the human-computer interaction.
It is easy to find enumerations of the supposed advantages of icons in popular literature.
Icons improve the productivity and reliability of work;
Icons are better than words for representing subtle visual and spatial concepts;
Icons save space;
Icons speed search;
Icons lead to immediate recognition;
Icons lead to better recall;
Icons reduce the necessity of reading;
All of these claims about icons implicitly compare icons to text in the interface. Many of the claims are psychological or perceptual in nature: that icons are easier to process than text.
NOT ALL ICONS ARE CREATED EQUAL
Icons differ in the degree of abstraction. Several icon classifications exist in the literature. Representational, abstract, and semi-abstract.
Representational (an trash can for "delete") Representational icons are simplified images of familiar objects or operations, for example, an image of a trash can to indicate a selective deletion operation. Their concreteness tend to be most effective since a small articulatory distance aids recognition. However, icons embodying an isolated analogy are often difficult for computer users to interpret, even concrete analogies, e.g. a wheelbarrow loaded with bricks to represent a move operation.
Abstract (a curved arrow for "reply") Abstract icons employ geometric shapes or graphic symbols instead of concrete images, for example, a curved arrow to represent “reply”. These often need to be learned, and when possible, existing conventions should be used, e.g. curved arrow pointing left for "reply", triangle pointing right for "play"
Semi-abstract (an arrow pointing to a folder for "move") Semi-abstract icons combine a representational pictorial element with an abstract symbol, for example, a folder with an arrow that indicates placing items in it.
Icon sets with more visual variety are easier to locate in a display
Visually simple icons work better than complex icons in icon search
Positional consistency in presentation of icons on the screen has a strong effect on usability and, once learned, helps compensate for initial differences in recognizability
Icons must be immediately recognizable to the targeted user population and use graphic conventions familiar to that group.
IMAGE SUPERIORITY EFFECT
When it comes to memory, researchers have known for more than 100 years that pictures and text follow very different rules. Put simply, the more visual the input becomes, the more likely it is to be recognized and recalled. The phenomenon is so pervasive, it has been given its own name: the pictorial superiority effect, or PSE.
All user interfaces make cognitive demands on users. Users must master special rules of system use, learn new concepts, and retain information in short-term memory. They must create and refine a mental model of how the system works and how they should use it. Successful user interface designs must respect the limitations of human cognitive processing.
Tests showed that visuals are processed 60,000 times faster than text, and people could remember more than 2,500 pictures with at least 90% accuracy several days post-exposure, even though subjects saw each picture for about 10 seconds.
The inefficiency of text has received particular attention. One of the reasons that text is less capable than pictures is that the brain sees words as lots of tiny pictures. And when we read, most of us try to visualize what the text is telling us. We essentially create icons inside our heads.
Using icons or symbols on a device serve as aids to memory, thereby reducing cognitive load since we can process icons and symbols much quicker than we can read text.
Graphics do what text alone cannot do. They quickly affect us both cognitively and emotionally:
Cognitively: Graphics expedite and increase our level of communication. They increase comprehension, recollection, and retention. Visual clues help us decode text and attract attention to information or direct attention increasing the likelihood that the audience will remember.
Emotionally: Graphics enhance or affect emotions and attitudes. They engage our imagination and heighten our creative thinking by stimulating other areas of our brain, which in turn leads to a more profound and accurate understanding of the presented material.
USEFULNESS OF ICONS
There are several reasons why the use of icons might have advantages over text in terms of human-computer interaction. It's been shown that recognition of visual images is superior to either recognition of words or sentences, and that humans have an almost unlimited ability to recognize images that they have seen before.
There are distinct resources available for different activities, such as perception and cognition. If the primary task is a problem solving task which requires cognitive resources, then the use of icons, which draw on a perceptual resource pool, may make more resources available for the primary task. A text-based interface, on the other hand, might compete with the primary task for cognitive resources.
Two kinds of perceptions are important in the acceptance of a system: perceived usefulness and perceived ease of use.
Perceived usefulness (PU) Perceived usefulness is the user’s subjective perception of the extent to which the system or software will aid in job performance. If you want to edit a movie and you open FinalCut Pro with all its movie editing related options you may perceive it as “useful”
Perceived ease of use (PEU) Perceived ease of use is the extent to which the user expects a system or software to require low effort to learn and use. When you open FinalCut Pro for the first time and you see all its movie editing related options, you may perceive it as “hard to use”.
The relationship of attitudes to the behavioral intention to use a system is that, all other things being equal, people will form an intention to use a system about which they have positive attitudes.
Icons give a positive early impression and thus support the formation of positive perceptions of ease of use and usefulness.
THE RESEARCH FINDINGS
Icons make systems appear more accessible to the new user may have the effect of increasing the perceived ease of use of a system and the user’s corresponding behavioral intention to learn and use it. With positive perceptions about a system, users are more willing to persevere in learning it in the face of any initial difficulties.
Icons with text labels were easier for participants to learn to recognize correctly but subsequently took more time for participants to choose from a display than unlabeled icons. However, it was also found that, once learned, there was little difference among icons in recognizability.
The combination of icons with text has the possibility of disambiguating the meaning of the menu choices available in the interface.
The results indicate a learning advantage for the label-only and icon + label interfaces compared to the icon-only interface for correctness, time and use of help.
On the other hand, the advantage in terms of correctness, although significant, was less exaggerated, suggesting that most participants could do most of the tasks regardless of the interface, simply by continuing to try different solutions until they were successful. The success of the label-only and icon + label groups suggests that in early learning text labels are more informative than icons.
The lack of difference between the label-only and icon + label groups in most comparisons suggests that the labels were the element that aided learning.
From these results, it appears that providing icons in combination with labels added little extra information of use in learning to manipulate the interface beyond what was conveyed by the labels.
A key question about these results is why the icon-only group performed so poorly. The point has been made in the past that icons, even representational icons, have to be learned. In early learning the meaning of the icons must be interpreted, and the meaning may not be obvious to the beginner from a pictorial representation alone.
Icon-only users have to first interpret the pictorial symbol then interpret its meaning in terms of system functionality. This dual burden is especially heavy in the case of learners who have low experience with both computers and the applications area. These learners have two disadvantages in interpreting icons. First, they do not known much about what the program they are learning is capable of doing. Without this knowledge, it is difficult for them to infer functionality from an icon alone. Second, they know little about the conventions of iconic representation.
For example, more experienced learners know that a trash can-like object is often used to represent deletions, a file folder-like icon to represent a container for documents, a house icon to represent the home screen etc. Experienced learners can interpret these objects when they learn new interfaces, even though the objects are slightly different in form from those they have previously encountered. Thus, for learners more experienced with computers and the special c application, we would expect better performance with the icon-only interface than was seen in this experiment. While experienced learners will still have to learn unfamiliar icons, there will be fewer such icons and the learners will have a better basis for interpretation because of their past experience.
Although the icon-only group learned more slowly, made more errors, and used more help in early blocks of trials in session one, it should be noted that they closed the gap relatively quickly. So the difficulties of interpretation of unlabeled icons can be overcome fairly rapidly.
The icon + label interface was always considered easier to use than either of the other interfaces, including the label-only interface. This is a case in which perceptions do not match performance, since the label-only group inconsistently performed as well as the icon-label group. It appears that an interface designed with icons suggests ease of use to the learner and is rated easier to use partially independent of performance.
The icon+ label and the label-only groups had an easier time getting the interface to do what they wanted in session one and, as a result, probably gained a better understanding of system functionality initially than did the icon-only group. So their perceptions of usefulness were formed at that early point and remained fairly stable as they gained more incremental skill in session two.
The results of this study lead to the conclusion that learners of application programs are aided in initial learning by the use of either icons with text labels or text labels alone.
Icons lacking labels performed very poorly in the early stages of learning, although learners of the icon-only interface caught up with the performance of the other two groups by the end of the first session.
Learners have less favorable perceptions of the usability and usefulness of text-only interfaces.
On the basis of our results we recommend the provision of text labels in initial use of iconic interfaces.
When words and icons are closely entwined, we augment intelligence by increasing ‘human bandwidth'—the capacity to take in, comprehend, and more efficiently synthesize large amounts of new information.
Learners who have little knowledge of the functionality of the application program and who have little knowledge of representational conventions often used in iconic interfaces are most in need of aid. These users have little basis for making inferences about the meaning of icons in the lack of text labels.
This suggests that the text labels play an important role briefly in the early stage of learning, but after that they lose their value. From this, it appears that labels may be suppressed after the users have worked with the program for a relatively short time.
Use icons whenever possible
Initially show labels, with an easy to locate option to turn them off.
On desktop software/web apps, show labels on hover (tooltips)
Show labels for the first few uses of the app, and once the user has tapped all or most icons in the app, hide the labels automatically and have it as an easy option to turn them back on.
The use of icons and labels in an end user application program: an empirical study of learning and retention. http://portfolio.educ.kent.edu/daltone/cmc2/articles/jp_use%20of%20icon%20labels_wiedenbeck.pdf
Pictorial Superiority Effect http://en.wikipedia.org/wiki/Picture_superiority_effect
Visual Memory http://en.wikipedia.org/wiki/Visual_memory
Optimizing Cognitive Load for Learning From Computer-Based Science Simulations http://steinhardtapps.es.its.nyu.edu/create/files/pubs/JEP_Lee_Plass_Homer_2006.pdf
Cognitive Load Theory http://www.southalabama.edu/oll/mobile/theory_workbook/cognitive_load_theory.htm
Cognitive load in reading a foreign language text with multimedia aids and the influence of verbal and spatial abilities http://steinhardtapps.es.its.nyu.edu/create/files/pubs/Plass%20et%20al.%202003.pdf
A study of short-term memory recall of pictures, words, and pictures and words presented together http://clearinghouse.missouriwestern.edu/manuscripts/473.php
Learning to order pictures and words: A model of sensory and semantic encoding. http://psycnet.apa.org/index.cfm?fa=buy.optionToBuy&uid=1978-11577-001