Paul N. Bennett

Researcher, Context, Learning, and User Experience for Search, Microsoft Research

Research Activities:

I am interested in the development, improvement, and analysis of machine learning methods with a focus on systems that can aid in the automatic analysis of natural language as components of adaptive systems or information retrieval systems. My current focus is on hierarchical and large-scale classification, search query classification and characterization, classification-aided IR ranking, diversity in retrieval results, preferences, and data mining via human computation applications.

My past work has examined a variety of areas — primarily text learning, ensemble methods, active learning, and obtaining reliable probability estimates, but also extending to machine translation, recommender systems, and knowledge bases.

Before coming to Microsoft, I obtained my Ph.D. from the Computer Science Department at Carnegie Mellon under Jaime Carbonell and John Lafferty.   Prior to that I worked closely with Ray Mooney and Robert Causey during my undergraduate days in the Computer Science, Philosophy, and Plan II Honors departments at the University of Texas at Austin.

I'm co-orgainzing a SIGIR 2009 Workshop on Redundancy, Diversity, and Interdependent Document Relevance.

Workshops, Collaborations and Papers:

• P.N. Bennett, N. Nguyen (2009).   Refined Experts: Improving Classification in Large Taxonomies.  In Proceedings of the 32nd Annual ACM SIGIR Conference (SIGIR 2009).  Boston, MA.  July 2009. (Talk powerpoint slides and PDF slide deck)
• K. Collins-Thompson, P.N. Bennett (2009).   Estimating Query Performance using Class Predictions..  Appears as a Poster-Paper in Proceedings of the 32nd Annual ACM SIGIR Conference (SIGIR 2009).  Boston, MA.  July 2009.
• P.N. Bennett, D.M. Chickering, A. Mityagin (2009).   Picture This: Preferences for Image Search..  Demo-paper in the Human Computation 2009 Workshop at KDD ’09 (see WWW'09 paper for primary version).  Paris, France.  June 2009.
• P.N. Bennett, D.M. Chickering, A. Mityagin (2009).   Learning Consensus Opinion: Mining Data from a Labeling Game.  In Proceedings of the 18th Annual International World Wide Web Conference (WWW '09).  Madrid, Spain.  April 2009.
• P.N. Bennett, B. Carterette, O. Chapelle, T. Joachims (2008).   Beyond Binary Relevance: Preferences, Diversity, and Set-Level Judgments, a summary of the SIGIR 2008 workshop.  In SIGIR Forum. pp 53-58. Dec 2008.
• B. Carterette, P.N. Bennett (2008).   Evaluation Measures for Preference Judgments .  Appears as a Poster-Paper in Proceedings of the 31st Annual International ACM SIGIR Conference on Research and Development in Information Retrieval.  Singapore.  July 2008.
• B. Carterette, P.N. Bennett, O. Chapelle (2008).   A Test Collection of Preference Judgments .  In Proceedings of the SIGIR 2008 Beyond Binary Relevance: Preferences, Diversity, and Set-Level Judgments Workshop .   Singapore.  July, 2008.
• B. Carterette, P.N. Bennett, D.M. Chickering, S.T. Dumais (2008).   Here or There: Preference Judgments for Relevance .  In Proceedings of ECIR 2008.  Glasgow, Scotland.  March-April 2008.
• P.N. Bennett (2007).   Neighborhood-Based Local Sensitivity .  In Proceedings of ECML 2007.  Warsaw, Poland.  September 2007.
• P. Donmez, J.G. Carbonell, and P.N. Bennett (2007).   Dual Strategy Active Learning .  In Proceedings of ECML 2007.  Warsaw, Poland.  September 2007.
• P.N. Bennett and J. Carbonell (2007).   Combining Probability-Based Rankers for Action-Item Detection.  In Proceedings of HLT-NAACL 2007.  Rochester, New York.  April 2007. (Talk powerpoint slides)
• P.N. Bennett, S.T. Dumais, and E. Horvitz (2005).  The Combination of Text Classifiers Using Reliability Indicators.  Information Retrieval.  8(1):67-100.  Kluwer Academic Publishers.  January, 2005. Initial Submission: 12/19/2002; Acceptance (Pending Revision): 7/21/2003; Final Version Acceptance: 1/6/2004.  (This paper revises and extends work originally presented in [Bennett, Dumais, & Horvitz; SIGIR 2002].)
• P.N. Bennett and J. Carbonell (2005).  Feature Representation for Effective Action-Item Detection. In the Beyond Bag-of-Words Workshop at the 28th Annual International ACM SIGIR Conference on Research and Development in Information Retrieval.  pp. 63-70.  Salvador, Brazil.  2005.  (This extends work presented as a SIGIR 2005 poster.)
• P.N. Bennett and J. Carbonell (2005).  Detecting Action-Items in E-mail.  Appears as a Poster-Paper in Proceedings of the 28th Annual International ACM SIGIR Conference on Research and Development in Information Retrieval.  Salvador, Brazil.  August 2005.
• R. Brown, R. Hutchinson, P.N. Bennett, J. Carbonell, P. Jansen (2003).  Reducing Boundary Friction Using Translation-Fragment Overlap.  In Proceedings of the Machine Translation Summit IX.  New Orleans, U.S.A.  September 2003.  (Shares significant overlap in content with publication listing for CMU-CS-03-138, 2003.)
• P.N. Bennett, S.T. Dumais, E. Horvitz (2003).  Inductive Transfer for Text Classification using Generalized Reliability Indicators. In Proceedings of the ICML-2003 Workshop on The Continuum from Labeled to Unlabeled Data.  Washington DC, U.S.A.  August 2003.
• P.N. Bennett (2003).  Using Asymmetric Distributions to Improve Text Classifier Probability Estimates.  In Proceedings of the 26th Annual International ACM SIGIR Conference on Research and Development in Information Retrieval.  Toronto, Canada.  July-August 2003.  See errata. (This paper revises work that originally appeared as CMU-CS-02-126.)
• P.N. Bennett, S.T. Dumais, and E. Horvitz (2002).  Probabilistic Combination of Text Classifiers Using Reliability Indicators: Models and Results.  In Proceedings of the 25th Annual International ACM SIGIR Conference on Research and Development in Information Retrieval, Tampere, Finland, August 2002. (An extended and revised version of this work appears in Information Retrieval, 2005.)
• R.J. Mooney, P.N. Bennett, and L. Roy (1998).  Book Recommending Using Text Categorization with Extracted Information.  In the AAAI-98/ICML-98 Workshop on Learning for Text Categorization and the AAAI-98 Workshop on Recommender Systems.  Madison, WI.  July 1998.
   

Tutorials:

• ICML 2009 Tutorial on Machine Learning in IR

Other Papers and Talks:

• J.M. Adams, P.N. Bennett, and A. Tomasic (2007).  Combining Personalized Agents to Improve Content-Based Recommendations.  CMU-LTI-07-015, Language Technologies Institute, School of Computer Science, Carnegie Mellon University, December 2007.
• P.N. Bennett (2006).  Building Reliable Metaclassifiers for Text Learning.  CMU-CS-06-121, Ph.D. Thesis, Computer Science Department, School of Computer Science, Carnegie Mellon University.  May 2006. See errata.
• Using Asymmetric Distributions to Improve Classifier Probability Estimates.  Presentation at the NIPS 2004 Workshop on Calibration and Probabilistic Prediction in Supervised Learning.
• Leveraging Locality for Text Classifier Combination.  Presentation at the IJCAI 2003 Doctoral Consortium.
• R. Hutchinson, P.N. Bennett, J. Carbonell, P. Jansen, R. Brown (2003).  Maximal Lattice Overlap in Example-Based Machine Translation.  CMU-CS-03-138, Computer Science Department, School of Computer Science, Carnegie Mellon University, June 2003.  (Also listed as CMU-LTI-03-174. Shares significant overlap in content with publication listing for MT Summit IX, 2003.)
• P.N. Bennett (2002).  Using Asymmetric Distributions to Improve Classifier Probabilities: A Comparison of New and Standard Parametric Methods.  CMU-CS-02-126, Computer Science Department, School of Computer Science, Carnegie Mellon University, April 2002.  See errata. (A revised version of this work appears in SIGIR 2003.)
• P.N. Bennett and A.D. Brown, eds. (2000).  The Guide to Living in Pittsburgh, 9th Edition, 299 pages, Pittsburgh, PA.  Dec/5, Inc.  2000.
• P.N. Bennett (2000).  Assessing the Calibration of Naive Bayes' Posterior Estimates.  CMU-CS-00-155, Computer Science Department, School of Computer Science, Carnegie Mellon University.  September 2000.
• P.N. Bennett (1998).  Text Categorization Through Probabilistic Learning: Applications to Recommender Systems. Undergraduate Honor Thesis, Department of Computer Sciences, University of Texas at Austin. May 1998. (Also appears as AI TR 98-270.)
   

Other Activities:

• Co-organizer of the KDD 2009 Workshop on Human Computation
• Co-organizer and Chair of the SIGIR 2008 Workshop on Beyond Binary Relevance: Preferences, Diversity, and Set-Level Judgments
• Publication Chair of ICML, 2006.
• Conference Reviewing:
  • Reviewer for AAAI Doctoral Consortium 2008-2009.
  • Reviewer for ACL 2008.
  • Reviewer for CHI 2009.
  • Reviewer for ECIR 2008-2009.
  • Reviewer for ECML 2008 Workshop on Preference Learning.
  • Reviewer for ECML 2009.
  • Reviewer for HLT-NAACL 2007.
  • Reviewer for SIGIR 2004-2009.
• Journal Reviewing
  • Reviewer for the Journal of Artificial Intelligence Research.
  • Reviewer for JMLR.
  • Reviewer for TOIS.
  • Reviewer for ACM Transactions on Internet Technology.
• Co-organizer IR Seminar Series, Summer 2004 - Summer 2005.