A Sensor Fusion Approach towards Gesture Recognition on the Wearable Ring Form Factor

Performing gesture recognition without infrastructure has been a long desired feature for seamless human computer interaction. In recent years, tablets and smartphones have made two-dimensional gestures a truly ubiquitous input modality. Yet, as computing becomes more pervasive, capacitive touch surfaces may not always be readily available or appropriate in many use case. Small form factor wearable computing devices that detect finger movements are an alternative way to approach this problem.

To address this, we developed a wearable platform that detects movements of a finger using audio and 3D acceleration. This platform converts finger movement data into a set of primitives that can be used for handwriting recognition or UI navigation in a context sensitive manner. To detect these gestures we use a sensor fusion approach that uses audio generated by surface friction to detect movement along a surface and acceleration to determine the direction of finger movement while in contact with the surface. By looking at the relationship between the envelope of audio emitted by surface friction and the acceleration from movement, we can disambiguate gesture data from spurious finger movements at low power cost. We implemented a platform prototype that collects audio and acceleration data and found our approach accurately detected gestures for several users. Our circuit design is amenable to small form factor implication with the end goal of a platform that can be worn as a ring.

Speaker Details

Jeremy Gummeson is a fifth year PhD student at the University of Massachusetts, Amherst where he is advised by Deepak Ganesan. His interest is in applying novel sensing and communication techniques to small form factor platforms that are powered primarily through a variety of energy harvesting techniques. He obtained his B.S. and M.S. in Computer Systems Engineering at the University of Massachusetts, Amherst.

Series:
Microsoft Research Talks
Date:
Speakers:
Jeremy Gummeson
Affiliation:
University of Massachusetts- Amherst

    • Portrait of Jeff Running

      Jeff Running

Series: Microsoft Research Talks