Share on Facebook Tweet on Twitter Share on LinkedIn Share by email
Geoffrey Zweig

Geoffrey Zweig

Research Areas

  • Machine Learning Methods for Automatic Speech Recognition
  • Voice Search: Getting information over the cellphone
  • Rapid Deployment of Dialog Systems
  • SCARF: Speech Recognition with Segmental Conditional Random Fields
  • Neural Network Language and Semantic Models


I am a Principal Researcher, and Manager of the Speech & Dialog research Group at Microsoft. I am responsible for building the next generation of speech recognition and dialog system technology. This covers work in neural network acoustic models, neural network language models, and active-learning methods for quickly building dialog systems. The work finds application in Microsoft products such as Cortana, Xbox, and the Project Oxford web services suite.

Recently I have contributed to two of the key Oxford services. LUIS (Language Understanding Intelligent Service) makes it very easy for a developer to add language understanding to applications. From a small number of examples, LUIS is able to determine a user's intent when they talk or type. CRIS (Custom Recognition Intelligent Service) provides companies with the ability to deploy customized speech recognition. The developer uploads sample audio files and transcriptions, and the recognizer is customized to the specific circumstances. This can make recognition far better in unusual circumstances, such as recognition on a factory floor, or outdoors. At runtime, both LUIS and CRIS are accessed via web APIs.

My personal research focuses on  improved algorithms for acoustic and language modeling for speech recognition, and language processing for downstream applications. Recent work has included the development of methods for conditioning recurrent neural networks on side-information for applications such as machine translation, and the use of recurrent neural network language models in first pass speech recognition. I also developed the Segmental Conditional Random Field (SCARF) toolkit  with Patrick Nguyen. Check out the SCARF project page. Recently I have begun to explore neural network models that encode semantic information, and some related publications are here. Prior to joining Microsoft, I worked at IBM Research for eight years, focusing on speech research, and most recently working on English, Arabic and Mandarin speech recognition systems for the DARPA EARS (Effective Affordable Reusable Speech-to-Text) and GALE (Global Autonomous Language Exploitation) programs.

I received my PhD in 1998 from the Computer Science Department of the University of California at Berkeley where I was advised by Stuart Russell and Nelson Morgan. I have served as Associate Editor of the IEEE Transactions on Audio Speech and Language and am currently on the editorial board of Computer Speech and Language. I am a member of the ACM and an IEEE Fellow.



I am an inventor of several patents, including:

  • U.S. Patent #8,145,484: Speech Processing with Predictive Language Modeling (2012)
  • U.S. Patent #7949526: Voice Aware Demographic Personalization (2011), with Y.C. Ju, A. Acero, and N. Bernstein.
  •  U.S. Patent #6,842,796: Information Extraction from Documents with Regular Expression Matching (2005), with M. Padmanabhan.
  • U.S. Patent #6,611,678: Device and Method for Trainable Radio Scanning (2003), with C. Neti.
  • U.S. Patent #6,411,933: Methods and Apparatus for Correlating Biometric Attributes and Biometric Attribute Production Features (2002), with S. Maes.
  • U.S. Patent #6,119,124: Method for Clustering Closely Resembling Data Objects (2000), with A. Broder, S. Glassman, M. Manasse, and G. Nelson.


E-mail: Geoffrey Zweig
U.S.Mail: Microsoft Corporation, One Microsoft Way, Redmond WA, 98052-6399, USA
Tel: (425) 421-6668
Fax: (425) 936-7329 (This is the main MS FAX number so make sure to send documents to my attention)