Mrs. B, a 63-year-old patient in the Memory Clinic and Memory Aids Clinic at Addenbrooke’s Hospital in Cambridge, England, had developed amnesia after sustaining a brain infection. So extensive was the loss of memory that within five days, she became completely unable to recall anything about a significant event unless she received memory assistance.

Researchers from Microsoft Research Cambridge had been aware of the work being done at Addenbrooke’s, renowned for its efforts to help patients experiencing a loss of memory. Microsoft Research Cambridge had developed a device called SenseCam, a wearable digital camera that takes photographs passively and automatically while being worn. Neuropsychologists at Addenbrooke’s were intrigued.

Mrs. B was given a SenseCam and was asked to wear it whenever she anticipated the sort of event she would like to remember. Having done so, she would spend an hour every two days reviewing the images the camera had taken.

This graph summarizes Mrs. B’s recall of autobiographical events. Three conditions are plotted: recall when no memory aid is used, the effect of reviewing a meticulous written diary, and what happens when SenseCam is used. After more than a month, Mrs. B recalled nothing of an event unless she previously had reviewed SenseCam images of that event, in which case she retained a remarkably complete memory that didn’t fade over time.

During the time Mrs. B reviewed the images, her memories of these significant events steadily increased. After two weeks, she could recall 90 percent of the events in question. Even more remarkably, after the trial concluded, she appeared to retain a lasting ability to recall the events—even without reviewing the images.

For those who work in assisting persons with extensive memory loss, the results were stunning, so much so that research is being extended to a number of projects in institutions around the world. The External Research & Programs group within Microsoft Research Redmond, Microsoft Research’s Technical Computing Initiative, and Microsoft Research Cambridge have contributed $550,000 and 50 of the devices—as well as expertise and support—to assist a number of associated projects.

A number of research questions still need to be answered before Microsoft can responsibly consider making SenseCam into a product available to the general public, but those who have worked with the SenseCam are its most effusive adherents.

Emma Berry was one of the Addenbrooke’s neuropsychologists assigned to the SenseCam collaboration. And from her experience, she is convinced that SenseCam can serve as a powerful memory aid, even for patients with severe memory impairment resulting from acquired brain injury and neurodegenerative disease.

“Not only does it allow people to recall memories while they are looking at the images, which in itself is wonderful,” says Berry, who now works with Georgina Browne, a fellow neuropsychologist at Addenbrooke’s, to further research on the technology, “but after an initial period of consolidation, it appears to lead to long-term retention of memories over many months, without the need to view the images repeatedly.

SenseCam's small size enables it to be worn around the neck without proving cumbersome or a nuisance to users.

“This is astonishing, given that the same people would ordinarily forget a significant, meaningful event within days of it happening. There is no other device, tool, or method that has the same effect. No wonder the memory community is so excited about this.”

Berry’s comments reflect the sort of enthusiasm SenseCam has engendered among clinicians who work with patients with memory impairment. The research and rehabilitation communities have been abuzz with the exciting potential the device holds and are eager to explore its potentially dramatic possibilities.

Following the success of the first trial, more were conducted, some with patients in the early stages of Alzheimer’s disease. Results have been promising, and Microsoft Research is eager to extend the work already under way.

“Microsoft Research develops applications and devices that are broadly usable—sometimes even beyond what we initially imagine, says Kristin Tolle, a program manager on the External Research & Programs team. “SenseCam is a good example. Initially, we thought of it as a general-purpose tool that aids in the capture of visual and sensor data. It was then theorized that it could potentially be used as a therapeutic device for patients suffering from a host of memory disorders. It made sense for us to turn to research experts in the field in order to conduct the relevant empirical studies.”

The field responded enthusiastically. The following projects are among those being funded by Microsoft:

• Adam Zeman, professor of Cognitive Neurology at the University of Exeter in the United Kingdom, is investigating how SenseCam might be able to help assess memory deficits and act as a memory aid for patients with a form of epileptic amnesia.
• Professor Phil Barnard, research scientist at the MRC Cognition and Brain Sciences Unit in Cambridge, is working with Dr. Linda Clare, reader in Psychology at the University of Bangor, Wales, to investigate how SenseCam might facilitate recollection in patients with dementia.
• Dr. Fergus Gracey and Dr. Andrew Bateman of The Oliver Zangwill Centre for Neuropsychological Rehabilitation in Ely, U.K., wants to use SenseCam images, combined with physiological sensor data, as a rehabilitation tool for patients with cognitive and memory impairment.
• Ronald Baecker, professor of Computer Science at the University of Toronto, is taking a look at the role SenseCam might be able to play in helping patients to reminisce.
• David Winkelaar of The Centennial Centre for Mental Health and Brain Injury, located in Ponoka, Alberta, will study the role of SenseCam as an aid to improve memory retention in patients with severe brain injuries.
• Professors Roberto Cabeza, of Duke University, and Martin Conway, of the University of Leeds, and Dr. Chris Moulin, also of the University of Leeds, are interested in why SenseCam works so powerfully and if it can lead to general improvements in memory. They plan to use behavioral and neuroimaging techniques to examine the effects on memory in healthy people, both young and old, who view SenseCam images.

“SenseCam images are one of the most potent cues to remembering I have ever encountered,” Conway says. “This is because they are similar to visual images in human memory in several respects. They are taken outside of conscious awareness and intentional control, just like the formation of human memories. They are fragmentary, essentially poor-quality records of experience, just what the brain needs as one of its main functions to fill in the gaps in input. They are visual, as is more than 80 percent of human memory. And because they are taken from the wearer’s own perspective, the SenseCam images correspond to external sensory changes. We think human memory might at least be partly triggered in this way.”

For Steve Hodges, principal hardware engineer for Microsoft Research Cambridge who has worked on SenseCam for a few years now, such testimonials are becoming happily common.

“The Sensors and Devices Group at Microsoft Research Cambridge is interested in a wide range of hardware-related technologies,” Hodges says. “SenseCam is an interesting project—at the core, the device is pretty simple, in terms of construction and operation, but when applied in the right way, you get results that are truly inspiring.”

SenseCam is the 1999 product of a concept from Lyndsay Williams, formerly of Microsoft Research Cambridge. A wide-angle lens affixed to a camera worn on a neck cord shoots photos from near eye-level, providing an approximation of what is in the field of vision of the person wearing the device. Worn this way, the camera is relatively stable, comfortable, and easy to put on and take off.

The SenseCam Viewer includes a set of standard video controls to enable to users to see a collection of captured images in a couple of speeds, in addition to other tools to help add, bookmark, and manipulate SenseCam-generated media.

Images taken by the SenseCam are stored as JPG files on a gigabyte-sized flash memory card that can hold more than 30,000 images—sufficient to capture a photo every 30 seconds for two waking weeks of a user’s life. The images can be downloaded to a PC and viewed using a VCR-type control that enables a videolike flip-book effect that can played at various speeds and can be rewound and paused. The SenseCam also is equipped with a variety of sensors that can be used to generate photos when environmental changes are detected.

Moulin was an early skeptic, but he has become an enthusiastic convert.

“My experience with SenseCam has been completely positive—and, at that, surprising,” he says. “I came on board feeling rather cynical, but I have completely revised that view.

“If SenseCam only gave you memories for things in the pictures,” he adds, “there would be no surprises, no magic, and no benefit to our research. What continues to amaze me, in my own personal use and in the SenseCam films we’ve made for others in the course of our research, is that SenseCam recovers memories for events, feelings, and details not captured in the lens and displayed on the screen. It’s not so much an aide-mémoire as a powerful cue for unlocking memories that you may have thought were lost.”

Browne, who has been working with Berry to study the use of SenseCam in those with Alzheimer’s disease, notes that not only does SenseCam help patients retain special memories, but it also fosters an increased sense of well-being and contentment.

“Perhaps most importantly,” she says, “what this has brought many of our patients is a better quality of life, an improvement to their self-esteem and day-to-day living as a result of increased confidence in their ability to retain recent memories.

“The work paves the way for assisted cognition—a term that has arisen out of the SenseCam work—as a viable mechanism for effective cognitive rehabilitation, specifically, the use of SenseCam as a rehabilitation tool in a variety of clinical populations and in the normal aging population, and generally, spurring the growth of a host of other assistive technologies and the development of new models of cognitive rehabilitation.”

Projects chosen for funding and assistance will be reviewed in June 2008 to assess their progress. Meanwhile, other potential applications are being explored, such as in a project called Participate—being conducted by the University of Nottingham, the University of Bath, the BBC, BT, Blast Theory, and ScienceScope and funded by the U.K.’s Department for Business, Enterprise and Regulatory Reform and the Engineering and Physical Sciences Research Council. Participate aims to design, develop, and test the utility of novel, pervasive, lightweight, and wearable technologies that support mass participation in science, education, art, and community life.

Other potential usages include:

• As a recording device for monitoring food intake, helping dieticians see the type, the quantity, and the timing of food a patient is eating.
• As a tool to assess accessibility issues encountered by wheelchair users.
• As a way to provide automatic generation of personalized memory experiences during visits to museums, national monuments, and other destinations.
• To coordinate disaster response by recording visual information encountered by those responding to disasters, people preoccupied with providing hands-on help.
• As an automatic diary that doesn’t require expensive, intrusive recording equipment or restrict a user’s activities.
• To monitor physiological data to help patients understand the sequence of events that precedes a period of intense anxiety or anger.
• To enable teachers to create a log of their day for post-instruction analysis.
• To monitor lighting conditions in schools and to learn how they affect students.
• As a choreographic tool that could enable efficient feedback for performers.

It’s a mind-boggling list, one certain to be extended as research continues. But for now, the focus is on memory retention.

“For the memory researcher,” Moulin says, “it is going to be an invaluable tool for rehabilitation and measurement. The most important memory to someone is for their daily lives and events. It has been difficult to measure, observe, and rehabilitate these, because the psychologist can’t follow the person around. SenseCam can go anywhere with the person and thus provides the researcher or clinician with a set of materials fit for rehabilitation and research.”

For Conway, SenseCam offers a couple of valuable avenues to explore.

“This comes down to two main interests for me,” he says. “One, SenseCam as a memory prosthesis for younger individuals, but most especially for older adults.

“Two, SenseCam as a tool for investigating naturally occurring memory or everyday memory. It is unique in this respect, and because an entire day’s worth of SenseCam pictures can be viewed in a short time, it is an entirely practicable tool and will, I predict, become a major enabling device in the study of human memory.”

The funding of further research to investigate such possibilities leaves those who have helped to improve and extend the device’s capabilities feeling exhilarated—perhaps even a bit awestruck.

“It’s clear that SenseCam has a huge potential as a memory aid,” Hodges says. “We’re really excited to be able to extend the project to actively involve so many leading researchers and clinicians around the world. In particular, we’re looking forward to learning more about what the technology can and can’t do—and possibly understand the mechanisms behind its effectiveness a little better.

“Ultimately, if we can conclusively demonstrate that SenseCam can improve memory, and thereby the quality of life, in patients with neurodegenerative diseases such as Alzheimer’s, that would be an absolutely tremendous result.”