Larry Heck

Larry Heck

Brief Bio

Larry Heck is a Distinguished Engineering in Microsoft Research. His research area is natural conversational interaction, focusing on open-domain NLP and dialog, machine learning, multimodal NUI, and inference/reasoning under uncertainty.  From 2005 to 2009, he was Vice President of Search & Advertising Sciences at Yahoo!, responsible for the creation, development, and deployment of the algorithms powering Yahoo! Search, Yahoo! Sponsored Search, Yahoo! Content Match, and Yahoo! display advertising. From 1998 to 2005, he was with Nuance Communications and served as Vice President of R&D, responsible for natural language processing, speech recognition, voice authentication, and text-to-speech synthesis technology. He began his career as a researcher at the Stanford Research Institute (1992-1998), initially in the field of acoustics and later in speech research with the Speech Technology and Research (STAR) Laboratory. Dr. Heck received the PhD in Electrical Engineering from the Georgia Institute of Technology in 1991.

Current Research Interests

  • Natural Language Processing (NLP): Open-Domain (Web-Scale)
  • Leveraging Knowledge Graphs for Unsupervised NLP
  • Deep Learning/Deep Neural Networks from Structured Knowledge
  • Multi-modal Situated Conversational Interactions
  • Inference and Reasoning

Work Experience

  • 2012 - Present:  Microsoft, Mountain View, CA
    Microsoft Research
    Distinguished Engineer
  • 2009 - 2012: Microsoft, Mountain View, CA
    Speech at Microsoft
    Chief Scientist
  • 2005-2009:  Yahoo, Sunnyvale, CA
    Head of Search and Advertising Sciences, Yahoo Labs
    Vice President
  • 1998-2005:  Nuance Communications, Menlo Park, CA
    Head of Nuance Research and Development
    Vice President
  • 1992-1998:  SRI International, Menlo Park, CA
    Speech Technology and Research (STAR) Lab
    Acoustics and Radar Technology Lab (ARTL)
    Senior Research Engineer


  • Georgia Institute of Technology
    PhD Electrical Engineering
  • Georgia Institute of Technology
    MS Electrical Engineering
  • Texas Tech University
    BS Electrical Engineering
Selected Publications
Early Deep Neural Network (Deep Learning)


  • L.P. Heck and M. Weintraub, "Method for establishing handset-dependent normalizing models for speaker recognition", US Patent #5950157, Sept. 1999.
  • L.P. Heck, "Voice authentication system having cognitive recall mechanism for password verification,” US Patent #6671672, Dec. 2003.
  • L.P. Heck and N. Mirghafori, "Method and system for on-line unsupervised adaptation in speaker verification,” US Patent #6,804,647, October 2004.
  • L.P. Heck, "Method and system for improving robustness in a speech system ,” US Patent #7203652, April 2007.
  • M.Cohen, L.P. Heck, J. Balogh, J. Riseman, N. Mirghafori, “Detection of Characteristics of Human-Machine Interactions for Dialog Customization and Analysis,” US Patent #7263489, August 2007.
  • L.P. Heck, “Adapting a Context-Independent Relevance Function for Identifying Relevant Search Results,” US Patent Application, 2007.
  • L.P. Heck, “Automated Learning from a Question and Answering Network of Humans,” US Patent Application, 2007.
  • L.P. Heck “Modeling, Detecting, and Predicting User Behavior with Hidden Markov Models,” US Patent Application, 2007.
  • K. Chen, R. Lu, C.K. Wong, G. Sun, L.P. Heck, B. Tseng, “Methods and Apparatus for Adapting a Ranking Function of a Search Engine For Use with a Specific Domain,” US Patent Application, 2008.
  • Augmented Conversational Understanding Agent. U.S. Patent No. 20,120,253,788. 4 Oct. 2012 (with Madhu Chinthakunta, David Mitby, and Lisa Stifelman)
  • Augmented Conversational Understanding Architecture. U.S. Patent No. 20,120,254,227. 4 Oct. 2012 (with Madhu Chinthakunta, David Mitby, and Lisa Stifelman)
  • Prosodic and Lexical Addressee Detection. MS# 337999.01, filed on 1/31/2013. (with Elizabeth Shriberg, Andreas Stolcke, and Larry Heck)
  • Using Human Wizards in A Conversational Understanding System. MS# 338086.01, filed on 3/4/2013. (with Lisa Stifelman, Madhu Chinthakunta, and Larry Heck)
  • Understanding Computer-Directed Utterances in Multi-User Dialog Systems, patent submission in progress. (with Dong Wang, Gokhan Tur, and Larry Heck)
  • Task Driven User Intents. U.S. Patent No. 20,120,253,791. 4 Oct. 2012 (with Madhu Chinthakunta, David Mitby, and Lisa Stifelman)
  • Sentence Simplification for Spoken Language Understanding. U.S. Patent No. 20,120,290,290. 15 Nov. 2012 (with Gokhan Tur, Dilek Hakkani-Tür, S Parthasarathy)
  • Personalization of Queries, Conversations, and Searches." U.S. Patent No. 2012135791. 5 Oct. 2012 (with Madhu Chinthakunta, David Mitby, and Lisa Stifelman)
  • Training Statistical Dialog Managers in Spoken Dialog Systems with Web Data. U.S. Patent No. 20,120,290,509. 15 Nov. 2012 (with Dilek Hakkani-Tür, Rukmini Iyer, and Gokhan Tur)
  • Location-Based Conversational Understanding, U.S. Patent No. 2012135210. 5 Oct. 2012 (with Madhu Chinthakunta, David Mitby, and Lisa Stifelman)
  • Conversational Dialog Learning and Correction, U.S. Patent No. 2012135229. 5 Oct. 2012 (with Madhu Chinthakunta, David Mitby, and Lisa Stifelman)
  • Translating Natural Language Utterances to Keyword Search Queries, MS #336777.01 filed on 6/2012 (with Dilek Hakkani-Tür, Gokhan Tur, and Rukmini Iyer)
  • Collaborative Learning through User Generated Knowledge, MS#337949.01 filed on 1/17/2013.
  • Probability-Based State Modification for Query Dialogues, MS#336765.01 filed on 12/14/2012 (with Dilek Hakkani-Tür, Gokhan Tur
  • Combined Activation for Natural User Interface Systems. U.S. Patent No. 20,120,254,810. 4 Oct. 2012 (with Madhu Chinthakunta, David Mitby, and Lisa Stifelman)
  • Contextual Query Adjustments Using Natural Action Input. MS# 336528.01 filed on 7/15/2012 (with Madhu Chinthakunta and Rukmini Iyer).
  • Exploiting Query Click Logs for Domain Detection in Spoken Language Understanding. U.S. Patent No. 20,120,290,293. 15 Nov. 2012 (with Dilek Hakkani-Tür and Gokhan Tur)
  • Exploiting the Semantic Web for Unsupervised Natural Language Semantic Parsing. MS# 337845.01, filed on 2/27/2013. (with Gokhan Tur, Minwoo Jeong, and Ye-Yi Wang)
  • Exploiting the Semantic Web for Unsupervised Spoken Language Understanding. MS# 338085.01, filed on 2/18/2013. (with Dilek Hakkani-Tür and Gokhan Tur)
  • Unsupervised Intent and Relation Detection using Knowledge Graph and Query Click Logs (with Larry Heck and Gokhan Tur)

Contact Info:
1288 Pear Avenue
Mountain View, CA 94043