Linda Lisle, Jianming Dong, and Scott Isensee
lalisle@us.ibm.com,
jdong@us.ibm.com,
isensee@us.ibm.com
IBM Corporation, Austin, Texas, USA
Abstract
This article discusses the evolution of a Web site as it grew through increases in content, changes in technology, and revisions to requirements. We summarize the history of the site and point out what we learned along the way. Although our case history is of a single site, we have experience developing a number of other sites and we believe the lessons discussed here are broadly relevant.
Purposes of the site
The site was designed and implemented by the corporate User Interface Architecture and Design group at IBM. We develop new user interface design techniques then spread this knowledge throughout the company and the industry through development of guidelines, tools, and methodologies.
Our department is a multidisciplinary group of user interface designers, writers, visual designers, and programmers. Department members have picked up the necessary Web site design skills as we have gone along and we have contracted for outside help in a few cases where skills shortages or workload required.
The Web site we report on here and a sister site we have developed for our corporate intranet, have the following purposes:
Evolution of the Human-Computer Interaction Web Sites
HCI Web Site Version 1
The first HCI (Human-Computer Interaction) Web site was released July, 1996. The primary goal of the site was to provide information and tools for user interface design. The audiences this information was targeted to were software designers and developers. As part of the main IBM Web site, however, we expected people with a broad range of backgrounds would visit our HCI site. To appeal to this broad audience we created a section of the site which provided an introduction to HCI. On the home page of the site we directed users to the branch of the site which would most interest them according to their background.
The site was designed to minimize unnecessary navigation so that users could get to content quickly and there would be minimal confusion about the structure of the site. There was a homepage, crossroads page and then content.
Home page of 1st HCI site with links for audience type
Crossroads page for choosing topic areas.
HCI Web Site Version 2
Immediately following the release of the first HCI Web site, work began on the second version of the site. This was released in November 1996. User feedback indicated that our site should be segmented by content groupings rather than by audience. Users generally did not like being labeled and they often found that both categories contained information of interest to them so we discontinued using audience categories to direct users to specific content areas.
In addition to major releases of the site, we made small additions of content on an ongoing basis. We noted that our Web site was drawing repeat visitors, but it wasn't clear to them when or where new content had been added. To address this problem, we added a category on the home page where we featured new additions to the site and added a category called Exhibits to highlight important new products.
The visual design was more sophisticated in this release and was very well received. The naming structure of the original Web site was carried over and built upon so that anyone who may have bookmarked the Web site would not receive a broken link. We use Shockwave to actively demonstrate some of our products and animated gifs were used to support the visual metaphor of "HCI Works". In the following figure of the HCI site home page, hammers pounded, hands moved, electricity zapped, etc.
Home page of second HCI Web site using animated gifs
The following is a screen capture of the RealPhone demonstrating the slide-out drawer using Shockwave. Users could interact with the demo such as opening and closing the drawer, lifting the receiver off the hook, etc. When users clicked on the handset, they heard a dial tone. This demonstrates many of the behaviors of the RealPhone product which users can download and use as a software telephone on their own computer.
Shockwave demo of IBM RealPhone
Feedback indicates that the Human-Computer Interaction web site is and was an overwhelming success. For instance:
I have just been directed to your web pages following from some comments I made about my frustrations with most mainstream GUIs on desktop computers. The work you are doing is great! I am fascinated in the ideas, as I have been following UI work as a hobby for a few years now. The ideas here are a breath of fresh air in a world of limiting, frustrating UIs for many computer users, new and old (I teach Mac computer skills to beginners).
More interesting though is that most feedback received was to engage us in helping the sender in finding information, or assisting them in an endeavor:
subject : proposed ISO standard 13407
comment : Is there a possibility to get the latest version of the proposed standard? If yes, where?
This is a clear indication that the interface was transparent to users and facilitated their willingness to engage with us in the business of Human Computer Interaction. Constructive critism was also appreciated and changes were made to the site as deemed necessary by the comments received.
Meanwhile, department members conducted research to find out users' models of the Web so that we could refine our designs and accomodate users' expections of our Web site. Some of the findings indicated users get lost very easily on the Web (we call it "lost in hyperspace") and have no sense of place. It was sometimes hard for them to determine when they had left a Web site and how they arrived at the next site. The findings from this research were used in developing further iterations of our Web site.
At the same time our departmental mission was evolving away from pure Human Computer Interaction to supporting the deployment of Ease of Use in all aspects of the IBM company. Therefore, the next version web site we created was called the Ease of Use Web Site. The first Ease of Use web site was developed to communicate our efforts to the press, influencers, and decision makers.
Ease of Use Web Site
IBM products were not perceived as easy to use. We were making significant efforts in the ease of use area and products that had benefitted from these efforts were beginning to hit the marketplace. Since this was a different audience than for our HCI site, we made it a separate site.
In July 1997, we released the first Ease of Use Web site. The intent of this Web site was to communicate simplicity. This Web site was very easy to navigate through and communicated the term "Easy" throughout.
We started to use Java at this point. We wrote Java applets which provided new controls and ease of use features for Web development. We made these applets available for download on our site. We also used Java to make the site more interactive. Sometimes the purpose was ease of use and sometimes entertainment. For example, we added a puzzle, implemented in Java, at the bottom of the home page. The user could move the squares around to solve the puzzle.
Home page for Ease of Use Site Version 1
This web site was effectively a "place holder" for the upcoming version released in December 1997. We did not receive any feedback about this web site and used our resources to focus on building a new Ease of Use web site that explored a "bring to" user model as well as researching the effects of using latest technologies.
Ease of Use Web Site Version 2
In December 1997, we redesigned the site to establish a sense of place and give the perception that information was being brought to the user. This was to address a problem found in usability testing performed on our site and other sites as well. In most Web sites, the user jumps from one page to the next with little commonality or context to provide a sense of place. They become lost in hyperspace.
New technologies were evolving rapidly on the Web. One of our missions was to try out these new technologies to learn and demonstrate how they should be used. This iteration of the site used a sophisticated mix of JavaScript, Java, Flash, CSS, and other technologies. Use of these technologies meant that users had to have the latest level browsers in order to see and use all that we designed. For users with lower level browsers, we provided a more basic version of the site. Due to resource constraints, however, this basic version did not include all of the content that the primary version of the site provided. We are looking for ways to deliver multiple versions of the site based on a single set of content in the future.
The audience definition was greatly expanded to include several specific audience segments. These were: web developers, application developers, human factors experts, information developers, students/professors, executives, press, and influencers. As each portion of the site was developed, we identified who it was targeted for and made sure it met the needs of that audience(s).
We felt our new interface made it practical to address the needs of multiple audience segments so we decided to merge the HCI and Ease of Use sites. The merger was partially completed this release and will be finished in the next release.
Home page for Ease of Use Web Site 2
This Web site was designed for 800 x 600 display resolution since the majority of our users were running this resolution or higher. It used frames for layout incorporating JavaScript to assist bringing information to the user and ensure the contents of each frame were coordinated.
In the Gallery section, Flash was used to present users with teasers allowing them to "experience" products and be entertained. Flash provides animated sequences that are streamed to the user's machine so that the interaction is fast and smooth. We created commercial-like animations describing the product we were demonstrating in very artistic, abstract ways to entertain users and capture their attention.
Representative frames from one of the animations on the Web site
This Web site was designed so that users never felt like they are leaving this "place". Information was brought to them. This was a difficult requirement to fulfill given the technologies available. We had to do a significant amount of code development and testing.
Object Oriented Web Design Techniques
During development of these sites, we have been evolving our design process and have applied object oriented design techniques. The techniques used here were derived from our OVID (Object, View, and Interaction Design) methodology. OVID was originally developed for GUI design but applies well to Web design. To learn more about OVID, see the paper at http://www.ibm.com/IBM/HCI/guidelines/design/ovidintro.html .
This methodology allows you to define objects in a Web site and the relationships between those objects. The following diagrams from development of the Ease of Use site show how we documented object relationships, views, and states.
Diagram of the objects and their views within the Gallery section.
Diagram of the states of an article object
These techniques allowed us to work out the structure and conceptual design of the site before worrying about the implementation details much the way an artist makes a sketch before putting paint on canvas.
User Feedback Techniques for the Web
Cognitive walkthrough and heuristic analysis
Cognitive walkthrough and heuristic analysis has been widely utilized in both conceptual design and requirements validations of human-computer interface design. During cognitive walkthrough evaluation, each screen of the presentation is reviewed using usability criteria. The goals for the evaluation are to make sure that users can use the presented information to make correct judgments of their actions and interpret the system feedback. In the conceptual design stage, review of screen layouts can generate more accurate presentation of information flow such as detailed information flow presentations such as object state transition charts. In the requirement validation stage, interface prototypes are evaluated to ensure that the specifications should be fulfilled with the existing design.
Heuristic analysis is more useful in the later stages of software design, especially after the prototype has been well developed. Design experts would use their web design experience and guidelines to provide suggestions. Consequently any issue can be addressed in this stage. Design revisions would be directly reflected in the final product.
In contrast to other application software, web sites have developed as hypermedia entities, which include hyperlinks via text, graphics, video, sound, and so on. Consequently, web usability testing is more concentrated on page design, navigation performance, information capacity, search mechanisms, multimedia implementation, platform support, etc.
Cognitive walkthrough methods were used to identify the potential problems for first-time users of the Ease of Use web site. We developed guidelines based on our experience developing and testing web sites. These web design guidelines are documented in the "Design" section of the Ease of Use site (http://www.ibm.com/ibm/easy/). They provide guidance to the planning, design, production and maintenance of web sites.
Formal usability testing
Formal usability tests are usually utilized to evaluate critical design features or evaluate product at a key point in design. Web site usability tests can be conducted off-line or on-line. The off-line test uses the downloaded web site in the local computer storage devices. This type of test enables the user to navigate in a designated scope of web pages without the impact of network traffic. Consequently, off-line web site testing has higher reliability and repeatability than on-line test. On the other hand, on-line tests can include deviations from the real world user performance which is often highly affected by network speed. It is hard to get accurate timing of user's performance time if there is large variation of network speed.
Formal usability test procedures for Web sites are similar to those for conventional software applications. Test participants are required to perform certain tasks using the Web site. Measurements of these usability tests often include performance time, errors, satisfaction, memory recall, etc.
On-line questionnaire
After the Web site has been released an on-line questionnaire is another channel of getting user feedback. This tool has been implemented in various IBM Web sites. Instead of acquiring design comments from a fixed group of users or design team, this method encourages participation from all users. Although the data would normally not fit into a clearly defined experimental matrix, the design issues addressed by voluntary participants are often valuable in heuristic evaluations.
In order to attract user participation and recruit participants for further usability testing, we held a contest. We placed a form on the web site where readers can register to win an IBM Thinkpad computer in return for agreeing to put their name on a list of potential test participants.
Much feedback has been received from on-line questionnaires. The questionnaire for the HCI site is posted at "http://www.ibm.com/ibm/hci/feedback/feedback.html". The questionnaire for the Ease of Use web site is available by following the Feedback link at "http://www.ibm.com/ibm/easy/". This feedback has been stored in a Lotus Notes database which can be accessed by Web site designers. Review of this user feedback has provided very valuable information for maintaining the current site and developing the new site.
Card Sorting and Cluster Analysis
In Web site design, it is essential to build up meaningful structure of information organization to ensure efficient navigation (Shneiderman 1992). The optimal information structure should reflect the purpose of the web site in a way that fits the users' mental model. This goal can be achieved from the collaboration of Web site designers and users.
The card sorting method is a tool to determine the structure of information organization. The following description outlines how we use the card sort method in Web site design. Based on the results of task analysis, the web site designer prepares a deck of index cards which represent the contents of each web page or the content to be placed on the Web. These cards are randomly ordered and provided to the participants representing the target user groups. Each participant is tested in an individual session to assure that responses will not be influenced by those of other participants. The participants are required to arrange the cards into groups according to the logical relationships. When they complete their reorganization, each resulting card stack is stapled. Then they are asked to arrange the original groups into large groups and suggest names for these large groups. The card sorting results are used to generate a similarity matrix, which is the basis of cluster analysis.
Card sorting technique has been successfully utilized in the information design of the overall IBM Web site (http://www.ibm.com) and its sub-sites. After cluster analysis, hierarchical structures of Web pages have been proposed. User-generated group names has also resulted from this.
Example of results from cluster analysis of card sort data
Usability evaluations we have conducted
Formal usability testing for the important design features has been conducted in all phases of the development process. For example, the "Design" section of the site has utilized a notebook and folder metaphor as information container. Although previous research has suggested that using such a metaphor would yield better navigation performance than the conventional scroll based view in a web page (Ueno, Suzuki and Ozawa, 1997), it is also important to find out users' reaction to this. The result of the test indicated that users' performance has been improved after using this metaphor.
Sample screen of "Design" section of IBM Ease of Use web site
One of our goals with the Ease of Use Web site is to demonstrate the newest Web design concepts and technologies. Information in different sections of the site has been provided in different styles according to the audiences. For instance, a tree structure presentation has been used in "Resources" which is intended for Web developers.
Sample screen of "Resources" section of IBM Ease of Use web site
On-line survey questionnaire technology was implemented. Survey questionnaire designers had access to a variety of questionnaire templates based on different testing purposes. The new questionnaire could easily be generated based on these questionnaire templates. After the questionnaire was finished, it was sent to a webmaster and posted at the desired URL. Software was developed to handle the data from the survey on a real time basis. The survey data was automatically summarized in tables and bar-charts. Participant background information is also presented. With the aid of this tool, user feedback information was processed immediately. Consequently, the cycle time of incorporating user feedback into new design is greatly reduced. We are currently working on a tool which automatically builds and posts the survey from a list of supplied questions. User surveys generally give the sites high marks, but always provide suggestions for further improvement.
Further research plans
The new Ease of Use Web site serves as an information resource on ease of use for a diverse audience. The information structures and presentation have been changed greatly compared to the old HCI site. In view of the goal of combining the HCI site with the new Ease of Use site, comparative evaluation of these sites will be conducted.
We will develop a list of the most common problems people have in using Web sites. We will then systematically propose solutions to these problems, prototype these design ideas, and test them. The successful design ideas will be reported on our Web site. Tools and/or guidelines for implementing these solutions will be provided.
We will continue to refine our understanding of the user model for the web. The better we understand the users, the better we will be able to develop interfaces which work for them.
We will develop and evaluate new design methodologies to make web user interface development more efficient and effective.
Lessons learned
Through our experience developing these Web sites, research we have conducted, feedback we've received, literature reviews, interviews with Web site developers, and related activities we have learned a number of valuable lessons. The following summary discusses the most important of those lessons.
Problems with browser versions
Netscape and Internet Explorer differ in many respects. You may want to support other browsers as well. For anything but a very simple Web page, you will have to go through contortions if you want the page to display consistently. Don't underestimate the amount of time it will take. It sometimes requires that you create parallel sets of pages for different browsers or add conditional code. You can use browser sniffer code to detect what browser the user has.
Browsers are continually adding new features and users do not always upgrade quickly. Using the new features can lock out some of your users. If you don't want to create multiple versions of your site you will have to trade off the benefits of using features available only with the newer browsers against reaching a wider audience.
Problems choosing and implementing a technology
You are working on shifting sand when designing for the Web. New technologies are coming along all the time. Some technologies (e.g. layers) are not accepted by all the major players and become obsolete quickly. If you choose the wrong technology, you will have to replace it quickly. Even if you choose the best technology currently available, it is still likely to be superseded before long. The speed at which you have to track changes is greater on the Web than it was for traditional applications. You also have less control. For example, unless you are writing intranet applications, you typically cannot control the environment in which your application will be run and displayed (i.e. browser level, plug-ins, operating system).
Tension between leading edge UI and simplicity
To improve interfaces, you have to do new things. New is inconsistent with old so users must learn. The new interface must be better enough than the old to justify the work of learning it. New UI techniques are often enabled by new implementation technologies which may not be accessible to all users.
Project management
The difficulty of developing for the Web is increasing. This requires more people with greater ranges of skills. The activities of these people must be coordinated. Classical project management techniques such as Gant charts have proven very helpful. The Web provides a great communication vehicle for the project members. Create a Web site where they can share project information.
Example of a Gant chart for Web site development
Dos and don'ts
Through experience, testing, and user feedback, we learned what works and what does not. To institutionalize this knowledge for ourselves and others, we created a set of web design guidelines which are available at http://www.ibm.com/ibm/easy/
IBM Web design guidelines
New concepts
By identifying user problems then implementing and testing potential solutions, we are developing UI techniques to improve the Web. For example, users would become "lost in hyperspace" jumping from link to link. We changed this experience from one of going from place to place to one where information is brought to the user in a single place. We accomplished this by using frames to reserve parts of the page for navigation and refreshing only the content frame as the user navigated throughout the site.
Example of a page design which establishes a sense of place
Synchronization of Multimedia
Synchronization of media is particularly difficult on the Web. Issues include speed of the processor on the client, speed of the internet connection, and load on the server. These unknowns make it difficult to select media attributes such as frame rates for video or animation and sampling rates for audio quality. Multimedia is not yet a standard element of the Web. It typically requires a plug-in.
A representative page on our Web site using multimedia
References
Roberts, D., Berry, R., Isensee, S. and Mullaly, J. (in press) Object, View, and Interaction Design, Toronto: IBM Press.
Shneiderman B.(1992). Designing the User Interface: Strategies for Effective Human-Computer Interaction, 2nd edn. Reading MA: Addison-Wesley
Ueno K., Suzuki K. and Ozawa H. (1997). Book based view and scrool based view: Which is suited for HTML viewers?, p795-798, in Design of Computing Systems: Social and Ergonomic Considerations, M.J. Smith, G. Salvendy and R. Koubek (Editors), Elsevier Science B.V., Amsterdam, The Netherlands.