During the design and implementation of the Task Gallery, we gathered empirical evidence to support our design decisions. Our first two studies examined task management with the 3D user interface before and after various usability issues were resolved.

We asked users to perform three types of activities in the Task Gallery: information management, spatial representations, and organizational strategies. Information management tasks included asking users to create, manipulate, and find tasks arranged in the Task Gallery. Spatial representation and organizational tasks involved asking users to orient themselves in the Task Gallery, organize their tasks spatially, and to draw their task layouts from memory at the session’s end. Methods used to examine subjects’ organizational strategies included looking at how long it took users to organize their tasks, as well as at the types and frequency of errors when asked to both find tasks in the environment and draw them from memory. We were interested specifically in how well users could create and modify tasks and the overall task space within the 3D Window Manager. In addition, detailed information about what kinds of organizing and retrieval strategies users employed was collected, so that we might be better able to support those strategies in future designs. We were especially interested in whether the organizing strategies were based on frequency, size, type of content, and time. Motivated by previous research suggesting that certain bodily axes are considered primary in the real world, (Siegel & White, *; Franklin & Tversky, 1990) we wanted to know if subjects’ organizing strategies and subsequent retrieval performance related to the up-down, front-back and left-right axes relative to the user’s orientation in space. For instance, do users prefer to store their tasks on the right-left walls of the hallway (the right-left axis), as previous work might suggest? Are there more retrieval errors within this axis, in that subjects easily confuse the right and left wall tasks with each other in memory? Of course, we were interested in what subjects’ spatial representation of the Task Gallery might be, and used memory measures to probe that information.

Secondary items of interest were whether it made any difference if the user created a task or if it was predetermined for them by the system, and how subjectively satisfying subjects considered the 3D user interface for task management overall.

Method

Subjects

Eleven subjects (five of them female)between the ages of 16-65 participated in two iterations of the same study. All were intermediate to expert Windows users. Five subjects participated in the first iteration evaluation of the prototype, and six subjects evaluated the second iteration prototype.

Materials

Materials for this study included an early prototype version of the 3D Window Manager we were designing, which included only “snapshots” of documents, not the actual application instances. However, this allowed subjects to manipulate the documents and tasks, arrange them in space. Eight tasks and their contents were identified prior to the study, based on commonly performed computer tasks observed during home visits. Tasks typically contained between 2 and 5 documents (like Word documents, Excel spreadsheets, web pages, email, etc.). Images of documents that would comprise these tasks were then saved away onto the “Data Mountain” in a default arrangement that was used for each subject. During the study, subjects went to the Data Mountain to get items to add to tasks and to create new tasks. There were 84 items on the Data Mountain at the beginning of each session.

The study was run on a 300Mhz Gateway Pentium computer with a (1024x768 resolution) display. A 15” NEC Mutisync LCD flat panel monitor was used as the display. Participants interacted with the software using a standard Microsoft serial mouse. No audio was included in the prototype.

Procedure

Participants carried out 6 tutorial trials, and 20 experimental trials. Four tasks were pre-arranged in the Task Gallery, and these tasks were used in the tutorial. In addition, the user created two new tasks and saved them during the tutorial. In the tutorial tasks, users were introduced to the concepts of navigating backwards and forwards through a 3D hallway, “pushing” documents away from them, “flicking” them to the side, and bringing them forward. Users also were introduced to the notion of arranging various documents into “tasks”, which could be saved away to a permanent spatial location (the floor, either wall of the hallway, or the ceiling). Once it was determined that the participants could perform all of the tutorial tasks easily, the experimental tasks were begun. No subjects took longer than a half hour of the 1.5-hour session in the tutorial. During the experimental tasks, users created two additional tasks, organized all the tasks in a way that was meaningful to them, retrieved eight tasks and their specific content items, and finally carried out various Windows operations. During the latter group of trials, users had to move documents toward and away from them, compare multiple documents at a time, and rearrange items within a task. After the first experimental trial, we asked users to draw what the hallway looked like to them, and what location and orientation they had within the hallway. At the end of the session, users drew their information layout in the hallway in as detailed a manner as they were able. In addition, they filled out a user satisfaction questionnaire. Following these last two activities, users were debriefed as to the goals of the study and given software gratuity for their participation.

Results

Results from both iterations of the prototype will be presented together, so that it is clear what impact the design changes had on performance.

Task Times

Task times for each subject were averaged across tasks for each task type in the experimental phase of the study. Overall task times improved after changes were made to the user interface by about 7 seconds, on average. For creating new tasks, average task time was reduced from 27.38 seconds to 2.2 seconds after the user interface changes. Subjects found all of the items required during the retrieval tasks. Therefore, only retrieval times are of interest. Average time to find tasks and their contents was reduced by about 9 seconds from the first iteration of the study (avg. task time=21.1 seconds) to the second (avg. task time=11.5 seconds). The time subjects spent managing windows in the user interface went from 46.7 seconds (iteration 1) to 36.8 seconds (iteration 2). None of these performance improvements reached statistical significance due to the small number of subjects and the large individual differences observed.

Spatial Representations

More interesting than task times were the findings with regard to subjects’ spatial representations of the Task Gallery. Most subjects thought of the hallway as a square, rectangle, or quadrilateral in shape, and all subjects guessed their face forward orientation accurately. This is not surprising, since subjects could only move forward and backwards in the hallway. Nevertheless, the pattern of results from the memory exercise was quite informative. Subjects’ drawings of the Task Gallery were scored for correct recall of tasks and correct placement of recalled tasks in their depictions of the space. 64% of the tasks were correctly recalled. Of those, 72% were drawn in the same position that they were placed in the Task Gallery. There were no differences between tasks that were “pre-arranged” in the space and those that the subjects created themselves.

Analysis of placement errors showed asymmetries with regard to how the subjects used and remembered different axes of the space. As shown in Table 2, subjects placed more tasks on the side walls than on the ceiling or floor of the Task Gallery. The vast majority of errors occurred when subjects drew tasks on the wrong surface (i.e., inter-surface errors). Drawing a task on the left wall when they had placed it on the right wall is an example of an inter-surface error. Displacement of tasks to either the left or right walls was more frequent than displacement to the ceiling or floor. In addition, memory was asymmetrical on this axis, with more tasks being missremembered as being on the left wall of the Task Gallery than the right. Forgetting the relative placement of tasks on the same wall was comparatively rare.

These data tentatively suggest that it was easier to remember depth relations between tasks on the same wall than it is to remember which wall a task was placed on. This is consistent with the literature on memory for spatial arrays, which finds that front-back relations are easier to represent than left-right relations. Most errors in placement involved displacement to the left or right, errors in front to back ordering on the same wall of the Task Gallery almost never occurred. The asymmetrical left-right bias could be due to a number of things including the fact that subjects glanced left to obtain documents while creating their tasks as well as our convention of reading from left to right.

Type of Asymmetry

Average Percent

Percent placement in space

Side walls

Ceiling and floor

52%

27%

Percent placement errors in drawings

Inter-surface errors

Displacement to side walls

Displacement to ceiling and floor

Intra-surface errors

92%

62%

18%

Results from spatial representation measure based on subjects’ drawings of task layouts in hallway.

User Satisfaction Ratings

Overall, user satisfaction ratings were quite high, given the early nature of the prototypes evaluated. Average satisfaction ratings were 4.9 for both the first and second iterations, using a 7 point scale, with 7=positive. Average response to the question as to whether or not users preferred this software over their current Windows desktop (an environment our test participants were extremely familiar with) favored the Task Gallery in both iterations of testing.

More Information

For more information, please refer to our paper which will be presented at CHI 2000.

Introduction