The Vocal Joystick: A Voice-Based Human-Computer Interface for Individuals with Motor Impairments

  • Xiao Li

Proceedings of the conference on Human Language Technology |

We present a novel voice-based humancomputer
interface designed to enable individuals
with motor impairments to use
vocal parameters for continuous control
tasks. Since discrete spoken commands
are ill-suited to such tasks, our interface
exploits a large set of continuous acousticphonetic
parameters like pitch, loudness,
vowel quality, etc. Their selection is optimized
with respect to automatic recognizability,
communication bandwidth, learnability,
suitability, and ease of use. Parameters
are extracted in real time, transformed
via adaptation and acceleration,
and converted into continuous control signals.
This paper describes the basic engine,
prototype applications (in particular,
voice-based web browsing and a controlled
trajectory-following task), and initial
user studies confirming the feasibility
of this technology.