The Analog Keyboard Project

Text input for small devices

As touch screens are getting smaller, soft keyboards are getting harder to use. For example, on a 1.6” smart watch, a soft keyboard with 10 keys across has keys less than 1/8” (3mm) wide. Speech recognition can be a viable alternative, but unfortunately, speaking into your watch is not always appropriate or even possible (noisy environments).

 

With the Analog Keyboard Project we are exploring handwriting recognition for text input on small touch screens. Handwriting, unlike speech, is discreet and not prone to background noise. And unlike soft keyboards, where many keys have to share the small touch surface, handwriting methods can offer the entire screen (or most of it) for each symbol. This allows each letter to be entered rather comfortably, even on small devices. In fact, it has been shown that some handwriting systems can be used without even looking at the screen [1]. Finally, handwriting interfaces require very little design changes to run on round displays, which are becoming increasingly popular.

 

Previous work

The idea to recognize handwritten letters on a watch screen is not new. In 1984 (!) Casio was selling a calculator watch that had a capacitive touch screen and recognized handwritten digits and math symbols. More information about this amazing device, the AT-550, can be found here.

 

We haven’t seen much of this technology in a long time, probably because calculator watches went out of style at some point during the 80s or 90s. But today—30 years after the Casio AT-550—as smart watches are gaining popularity, we think it is time to revisit this idea.

Prototype

To evaluate handwritten text input on current smart watches we have built a prototype for the Android Wear platform. This video shows our prototype running on both round and square Android Wear devices.

The prototype is available for download for research purposes. Before you use it please make sure to read the Usage tips and Limitations below. If you have additional questions or feedback, please let us know. Our contact email is shown at the end of the video. The project is led by Wolf Kienzle and Ken Hinckley.

 

Update: a number of readers have asked us whether this method is identical to Palm’s Graffiti. It is not. The key difference is that we let users write normal English letters, whereas Graffiti requires users to learn a custom gesture for each character.

Usage tips

·         Write lowercase. You can enter lowercase English letters (print), numbers, and basic punctuation symbols. Uppercase letters are not supported (see Limitations below).

·         Start slowly and neatly. When using the prototype for the first time, watch the fading strokes on the writing panel and make sure they are legible and properly separated from one another. The recognition accuracy should be fairly high that way. Then try to develop a sense for how sloppy you can write while still getting good accuracy.

 

·         Frequent words like “this”, “and”, etc. will often be auto-corrected by the system once all characters have been entered, so it’s generally faster to finish writing these words even if there are wrong letters along the way.

·         Rare words on the other hand, like names or hashtags, should be written slowly and verified character by character (auto-correct is suppressed if you pause a little after each character).

 

·         Eyes-free. This is fun to try: how long a word can you write without looking at the watch?

 

·         Backspace key has a repeat function (long-press)

 

Limitations

We are making this software publicly available in order to gather feedback from real-world users and scenarios (as opposed to a lab environment). It is free to use and should work with any Android Wear app that uses text input, like web browsers and message apps. However, please keep in mind that it is a research prototype, not a polished product. It comes with a number of technical limitations:

1.      It is not a store app. It needs to be side-loaded onto your watch using adb (Android Debug Bridge). The downloaded zip file contains a text file with instructions.

 

2.     It has been tested only on the Samsung Gear Live (square, 320x320 pixels) and the Moto 360 (round, 320x290 pixels). The 280x280 resolution or upcoming round watches are not supported at this time.

 

3.     For simplicity, all processing is done on the watch. This can be seen as a feature, since it works even when you’re not connected to your phone. However, it will also drain your battery (while in use) faster than you would expect from a keyboard.

 

4.     To reduce the computational load, uppercase letters are not supported in the current version. Similarly, our language and character models are trimmed down to fit on the watch. Overall recognition accuracy is decent now but will improve significantly once we offload some computation to the phone or a cloud service.

 

5.     Current Android watches typically have a single core processor. Even though we do all processing on background threads, the responsiveness of the UI is sometimes affected, especially if the host app (web browser, messenger) uses the CPU as well.

 

6.     There is a bug on the Moto 360. The Android OS shows the keyboard about 30 pixels above bottom of the screen, leaving a gap of precious screen space. We believe this is an Android bug [please contact us if you know anything about this issue]. Unfortunately, this shrinks the writing surface on the Moto 360. Also, strokes starting in that bottom area will not register.

 

7.     To comply with Microsoft Research release policies, our language model does not include (even mildly) offensive terms. Please do not try to test the system using profanities—it will not work well.

References

[1] Goldberg, D., & Richardson, C. Touch-typing with a stylus, CHI ‘93

 

Android is a trademark of Google Inc.

Contact us Privacy and Cookies Terms of use Trademarks   © 2014 Microsoft Corporation.     Microsoft