Presentation Notes from Microsoft Faculty Summit - Memex Meeting

Introduction
USC CNTV (Film School) - Interactive Media
Julian Bleecker
Chris Kyriakakis, IMSC

Audio is a neglected stepchild

Context Sensitive Audio Analysis for devices
Enable the elmo doll to wait for kids to stop talking
Enable the dolls to know the level of excitement in the group
Enable a cell phone to know when I am in a conversation, or driving, or in the bathroom

Context Sensitive Audio Analysis to Facilitate group meetings - (Stanford)
speaker id, speaker behavior, video steering

Interdisciplinary Balloon
The specifics are not as important, as the kinetic energy
My thoughts and observations in wearing a personal audio recorder

Our job is to 1) inspire each other (kinetic energy) and support each other (num of molc)

OBSERVATIONS REGARDING MY BEHAVIOR

I can not lose my hard-drive

Changes my Listening Behavior
Shut up!
Allows me to listen *to that person* better (cadence and style)
We both heard entirely different worlds
Student - fine tune my approach

Makes Me Shut Up
At Doctors - I want to get it all recorded

Feeling of Security
Go ahead - mug me
Speak out license plate
Perfect alibi - one can never say something happened that did not (leave the door cracked)

Phase Lock Loop of Life
Art piece - images of me on the freeway everyday
Mix what I say in the first 10 seconds after I leave the elevator at the club (ding)

Darn, I wish I recorded that
Knowing it is recorded allows for a relaxed concept-based conversation
Concept-based conversations versus fact-based conversation
You can relax and enjoy the conversation
Imy default is now that it *is* being recorded, so when it is not, it is a loss.
I can relax when listening to my mother telling me family memories

Allows more Partial Attention requires persistent recording
Nobody pays attention anyway
We half listen, then figure we can 'find it later'
This has extended into life
Text messaging during meetings
Checking email during my presentation

Nobody Can Lie
A verbal agreement will be just as good as a written one
But this is just not true
In fact, Audio only is a lie

LEGAL AND PRIVACY ISSUES

Security issues These are going to go away -

because everything is or will be recorded anyway
Reasonable expectation of privacy
Sharon Stone anecdote
You are always being recorded

Because the medical value is simply too high

Because we will just wear a button that says 'you are being recorded'

Because we can use the data for homeland security

Because speaker ID is possible - Just record the wearer -
Throat mike
Body facing mike, and lo frequency detection

ALTERNATIVE DEVICE SCENARIOS

'Active Sense Cam'
AugCogAudFob - can it do what Yyvone used to do?
Detect my 'state of mind' and then cue me when to do tasks

SenseCam as a Document Device
Travel to Japan - copy all my papers, then throw away

Automated Keystone Correction
Symbol recognition
Becomes a 'Document Device'
Can project a virtual 'frame' to know what it is recording

Networked SenseCams will happen
Audio world - can use the closest/best microphone

Stereoscopic SenseCam
Can pull out depth information
My Objects Stream

TeleSenseCam
Giving recorder to student to grab a lecture for me
Giving recorder to my daughter to hear her play from the stage and her friends

Personal Digital Surrogate (PDS)

DMV queing indicator

Type one: Secure, Signed, Personal Presence

Type two: cheap, o.k. to lose it

Send your surrogate by Fed-Ex
Know where your surrogate is (send a cell phone with GPS in it)
Use it for Share Holder meetings - it is a legal representative

Fed-back data - guarantee communication

 

LESSONS FROM THE WORLD OF SCRAPBOOKS

Many treat scrap books as the event - as the record of life
We should not be surprised by the memory aid value - pictures of vacations have always done this
This difference is that Imaging is dirt cheap
My students are doing this now - no need for the 'right shot'
Imaging is cheap

Scrap Books are Meant to Be Shared
Flickr images from school events
If you miss a party, you go check it out anyway

SenseCam might help us remember other's memories
But they also can help us remember other people's episodes

Can SenseCams be used to create false memories?
I would like to insert virtual characters within the sense cam data stream
Just enough for you to believe you were there, and then add a few twists
John Nordlinger points out that one can enhance memories (highlight objects perhaps)

Reality does not exist unless it has been packaged onto a hard-drive
For my students, Our life experience, our worlds are becoming more virtual
And I think that this is just a form of 'virtualizing' are real memeory

Recording a life - or editing a life?
Taking a photograph is not *recording*, it is *editing*
What you really want is to simply have everything recorded, then you can go back and 'take pictures' or edit, later.
To do this, context really helps, which can be had from other data types

(Vision algorithms for audio matching)

IT IS IRRESPONSIBLE TO NOT WEAR A SENSECAM

A Life Lived, A Life Erased (don't freeze your brain, record its behavior)
It is irresponsible to *not* wear one
"Papa, you didn't record your life?"
(just as lives do not exist unless a book was written, or a story remembered)

Collaborate with other SenseCam and MyLifeBits researchers to jointly create an architecture (DREAM) that coordinates cross-linking of data from a range of research efforts.

Personal persistent recording technologies see, feel, and hear everything in a user's day, with a concomitant need to make sense of all this sensory data - to sort it, file it, and put it into context. In order to achieve this, a great deal of post-activity processing must take place which is likely to take hours per day. This processing can be likened to what the human mind must do after a day full of sensory input - digest the data overnight to find links, associations, narratives, and meaning. In other words, SenseCams will need to dream at night, just like their owners. In order to enable such off-line processing, a number of different signal processing and context deriving algorithms will need to be invented and then integrated into a single architecture: a Data Recovery Engine for Augmented Memory, or a DREAM engine.
 

Posted by mbolas at July 19, 2006 04:52 PM