During the design and
implementation of the Task Gallery, we gathered empirical evidence to support
our design decisions. Our first two
studies examined task management with the 3D user interface before and after
various usability issues were resolved.
We asked users to perform three types of activities in the Task
Gallery: information management, spatial representations, and organizational
strategies. Information management
tasks included asking users to create, manipulate, and find tasks arranged in
the Task Gallery. Spatial
representation and organizational tasks involved asking users to orient
themselves in the Task Gallery, organize their tasks spatially, and to draw
their task layouts from memory at the session’s end. Methods used to examine subjects’
organizational strategies included looking at how long it took users to organize
their tasks, as well as at the types and frequency of errors when asked
to both find tasks in the environment and draw them from memory.
We were interested specifically in how well users could create and modify
tasks and the overall task space within the 3D Window Manager.
In addition, detailed information about what kinds of organizing and
retrieval strategies users employed was collected, so that we might be better
able to support those strategies in future designs.
We were especially interested in whether the organizing strategies
were based on frequency, size, type of content, and time.
Motivated by previous research suggesting that certain bodily axes are
considered primary in the real world, (Siegel & White, *; Franklin &
Tversky, 1990) we wanted to know if subjects’ organizing strategies and
subsequent retrieval performance related to the up-down, front-back and
left-right axes relative to the user’s orientation in space.
For instance, do users prefer to store their tasks on the right-left
walls of the hallway (the right-left axis), as previous work might suggest?
Are there more retrieval errors within this axis, in that subjects easily
confuse the right and left wall tasks with each other in memory?
Of course, we were interested in what subjects’ spatial representation
of the Task Gallery might be, and used memory measures to probe that
Secondary items of interest were whether it made
any difference if the user created a task or if it was predetermined for them by
the system, and how subjectively satisfying subjects considered the 3D user
interface for task management overall.
Eleven subjects (five of them female)between
the ages of 16-65 participated
in two iterations of the same study. All
were intermediate to expert Windows users. Five subjects participated in the first iteration evaluation
of the prototype, and six subjects evaluated the second iteration prototype.
Materials for this study included an early prototype
version of the 3D Window Manager we were designing, which included only
“snapshots” of documents, not the actual application instances.
However, this allowed subjects to manipulate the documents and tasks, arrange them in space. Eight
tasks and their contents were identified prior to the study, based on commonly
performed computer tasks observed during home visits. Tasks typically contained between 2
and 5 documents (like Word documents, Excel spreadsheets, web pages, email,
etc.). Images of documents
that would comprise these tasks were then saved away onto the “Data
Mountain” in a default arrangement that was used for each subject.
During the study, subjects went to the Data Mountain to get items to add
to tasks and to create new tasks. There
were 84 items on the Data Mountain at the beginning of each session.
The study was run on a 300Mhz Gateway Pentium computer
with a (1024x768 resolution) display. A
15” NEC Mutisync LCD flat panel monitor was used as the display.
Participants interacted with the software using a standard Microsoft
serial mouse. No audio was included
in the prototype.
Participants carried out 6 tutorial trials, and 20
experimental trials. Four tasks
were pre-arranged in the Task Gallery, and these tasks were used in the
tutorial. In addition, the user created two new tasks and saved them
during the tutorial. In the
tutorial tasks, users were introduced to the concepts of navigating backwards
and forwards through a 3D hallway, “pushing” documents away from them,
“flicking” them to the side, and bringing them forward.
Users also were introduced to the notion of arranging various documents
into “tasks”, which could be saved away to a permanent spatial location (the
floor, either wall of the hallway, or the ceiling). Once it was determined that the participants could perform
all of the tutorial tasks easily, the experimental tasks were begun.
No subjects took longer than a half hour of the 1.5-hour session in the
tutorial. During the experimental
tasks, users created two additional tasks, organized all the tasks in a way that
was meaningful to them, retrieved eight tasks and their specific content items,
and finally carried out various Windows operations.
During the latter group of trials, users had to move documents toward and
away from them, compare multiple documents at a time, and rearrange items within
a task. After the first
experimental trial, we asked users to draw what the hallway looked like to them,
and what location and orientation they had within the hallway.
At the end of the session, users drew their information layout in the
hallway in as detailed a manner as they were able. In addition, they filled out a user satisfaction
questionnaire. Following these last
two activities, users were debriefed as to the goals of the study and given
software gratuity for their participation.
Results from both iterations of the prototype will be
presented together, so that it is clear what impact the design changes had on
Task times for each subject were averaged across tasks for
each task type in the experimental phase of the study.
Overall task times improved after changes were made to the user interface
by about 7 seconds, on average. For
creating new tasks, average task time was reduced from 27.38 seconds to 2.2
seconds after the user interface changes. Subjects
found all of the items required during the retrieval tasks.
Therefore, only retrieval times are of interest.
Average time to find tasks and their contents was reduced by about 9
seconds from the first iteration of the study (avg. task time=21.1 seconds) to
the second (avg. task time=11.5 seconds).
The time subjects spent managing windows in the user interface went from
46.7 seconds (iteration 1) to 36.8 seconds (iteration 2).
None of these performance improvements reached statistical significance
due to the small number of subjects and the large individual differences
More interesting than task times were the findings with
regard to subjects’ spatial representations of the Task Gallery.
Most subjects thought of the hallway as a square, rectangle, or
quadrilateral in shape, and all subjects guessed their face forward orientation
accurately. This is not surprising,
since subjects could only move forward and backwards in the hallway.
Nevertheless, the pattern of results from the memory exercise was quite
informative. Subjects’ drawings
of the Task Gallery were scored for correct recall of tasks and correct
placement of recalled tasks in their depictions of the space. 64% of the tasks
were correctly recalled. Of those,
72% were drawn in the same position that they were placed in the Task Gallery.
There were no differences between tasks that were “pre-arranged” in the
space and those that the subjects created themselves.
Analysis of placement errors showed asymmetries with
regard to how the subjects used and remembered different axes of the space.
As shown in Table 2, subjects placed more tasks on the side walls than on
the ceiling or floor of the Task Gallery. The
vast majority of errors occurred when subjects drew tasks on the wrong surface
(i.e., inter-surface errors). Drawing a task on the left wall when they had
placed it on the right wall is an example of an inter-surface error.
Displacement of tasks to either the left or right walls was more frequent than
displacement to the ceiling or floor. In
addition, memory was asymmetrical on this axis, with more tasks being
missremembered as being on the left wall of the Task Gallery than the right.
Forgetting the relative placement of tasks on the same wall was
These data tentatively suggest that it was easier to
remember depth relations between tasks on the same wall than it is to remember
which wall a task was placed on. This is consistent with the literature on
memory for spatial arrays, which finds that front-back relations are easier to
represent than left-right relations. Most errors in placement involved displacement to the left or right, errors in
front to back ordering on the same wall of the Task Gallery almost never
occurred. The asymmetrical left-right bias could be due to a number of things
including the fact that subjects glanced left to obtain documents while creating
their tasks as well as our convention of reading from left to right.
Type of Asymmetry
placement in space
Ceiling and floor
placement errors in drawings
to side walls
to ceiling and floor
from spatial representation measure based on subjects’ drawings of
task layouts in hallway.
User Satisfaction Ratings
Overall, user satisfaction ratings were quite high, given
the early nature of the prototypes evaluated.
Average satisfaction ratings were 4.9 for both the first and second
iterations, using a 7 point scale, with 7=positive.
Average response to the question as to whether or not users preferred
this software over their current Windows desktop (an environment our test
participants were extremely familiar with) favored the Task Gallery in both
iterations of testing.
For more information, please refer to our paper
which will be presented at CHI