Paul N. Bennett

Researcher, Context, Learning, and User Experience for Search, Microsoft Research

Research Activities:

I am interested in all aspects of the development, improvement, and analysis of machine learning methods with a focus on systems that can aid in the automatic analysis of natural language as components of adaptive systems or information retrieval systems.  My work has examined a variety of areas — primarily text learning, ensemble methods, and obtaining reliable probability estimates, but also extending to machine translation, recommender systems, and knowledge bases.

Before coming to Microsoft, I obtained my Ph.D. from the Computer Science Department at Carnegie Mellon under Jaime Carbonell and John Lafferty.   Prior to that I worked closely with Ray Mooney and Robert Causey during my undergraduate days in the Computer Science, Philosophy, and Plan II Honors departments at the University of Texas at Austin.

I'm co-orgainzing a SIGIR 2009 Workshop on Redundancy, Diversity, and Interdependent Document Relevance and a KDD 2009 Workshop on Human Computation. I'm also co-presenting an ICML 2009 Tutorial on Machine Learning in IR.

Workshops, Collaborations and Papers:

• P.N. Bennett, D.M. Chickering, A. Mityagin (2009).   Learning Consensus Opinion: Mining Data from a Labeling Game.  To Appear in Proceedings of the 18th Annual International World Wide Web Conference (WWW '09).  Madrid, Spain.  April 2009.
• P.N. Bennett, B. Carterette, O. Chapelle, T. Joachims (2008).   Beyond Binary Relevance: Preferences, Diversity, and Set-Level Judgments, a summary of the SIGIR 2008 workshop.  In SIGIR Forum. pp 53-58. Dec 2008.
• B. Carterette, P.N. Bennett (2008).   Evaluation Measures for Preference Judgments .  Appears as a Poster-Paper in Proceedings of the 31st Annual International ACM SIGIR Conference on Research and Development in Information Retrieval.  Singapore.  July 2008.
• B. Carterette, P.N. Bennett, O. Chapelle (2008).   A Test Collection of Preference Judgments .  In Proceedings of the SIGIR 2008 Beyond Binary Relevance: Preferences, Diversity, and Set-Level Judgments Workshop .   Singapore.  July, 2008.
• B. Carterette, P.N. Bennett, D.M. Chickering, S.T. Dumais (2008).   Here or There: Preference Judgments for Relevance .  In Proceedings of ECIR 2008.  Glasgow, Scotland.  March-April 2008.
• P.N. Bennett (2007).   Neighborhood-Based Local Sensitivity .  In Proceedings of ECML 2007.  Warsaw, Poland.  September 2007.
• P. Donmez, J.G. Carbonell, and P.N. Bennett (2007).   Dual Strategy Active Learning .  In Proceedings of ECML 2007.  Warsaw, Poland.  September 2007.
• P.N. Bennett and J. Carbonell (2007).   Combining Probability-Based Rankers for Action-Item Detection.  In Proceedings of HLT-NAACL 2007.  Rochester, New York.  April 2007. (Talk powerpoint slides)
• P.N. Bennett, S.T. Dumais, and E. Horvitz (2005).  The Combination of Text Classifiers Using Reliability Indicators.  Information Retrieval.  8(1):67-100.  Kluwer Academic Publishers.  January, 2005. Initial Submission: 12/19/2002; Acceptance (Pending Revision): 7/21/2003; Final Version Acceptance: 1/6/2004.  (This paper revises and extends work originally presented in [Bennett, Dumais, & Horvitz; SIGIR 2002].)
• P.N. Bennett and J. Carbonell (2005).  Feature Representation for Effective Action-Item Detection. In the Beyond Bag-of-Words Workshop at the 28th Annual International ACM SIGIR Conference on Research and Development in Information Retrieval.  pp. 63-70.  Salvador, Brazil.  2005.  (This extends work presented as a SIGIR 2005 poster.)
• P.N. Bennett and J. Carbonell (2005).  Detecting Action-Items in E-mail.  Appears as a Poster-Paper in Proceedings of the 28th Annual International ACM SIGIR Conference on Research and Development in Information Retrieval.  Salvador, Brazil.  August 2005.
• R. Brown, R. Hutchinson, P.N. Bennett, J. Carbonell, P. Jansen (2003).  Reducing Boundary Friction Using Translation-Fragment Overlap.  In Proceedings of the Machine Translation Summit IX.  New Orleans, U.S.A.  September 2003.  (Shares significant overlap in content with publication listing for CMU-CS-03-138, 2003.)
• P.N. Bennett, S.T. Dumais, E. Horvitz (2003).  Inductive Transfer for Text Classification using Generalized Reliability Indicators. In Proceedings of the ICML-2003 Workshop on The Continuum from Labeled to Unlabeled Data.  Washington DC, U.S.A.  August 2003.
• P.N. Bennett (2003).  Using Asymmetric Distributions to Improve Text Classifier Probability Estimates.  In Proceedings of the 26th Annual International ACM SIGIR Conference on Research and Development in Information Retrieval.  Toronto, Canada.  July-August 2003.  See errata. (This paper revises work that originally appeared as CMU-CS-02-126.)
• P.N. Bennett, S.T. Dumais, and E. Horvitz (2002).  Probabilistic Combination of Text Classifiers Using Reliability Indicators: Models and Results.  In Proceedings of the 25th Annual International ACM SIGIR Conference on Research and Development in Information Retrieval, Tampere, Finland, August 2002. (An extended and revised version of this work appears in Information Retrieval, 2005.)
• R.J. Mooney, P.N. Bennett, and L. Roy (1998).  Book Recommending Using Text Categorization with Extracted Information.  In the AAAI-98/ICML-98 Workshop on Learning for Text Categorization and the AAAI-98 Workshop on Recommender Systems.  Madison, WI.  July 1998.
   

Other Papers and Talks:

• J.M. Adams, P.N. Bennett, and A. Tomasic (2007).  Combining Personalized Agents to Improve Content-Based Recommendations.  CMU-LTI-07-015, Language Technologies Institute, School of Computer Science, Carnegie Mellon University, December 2007.
• P.N. Bennett (2006).  Building Reliable Metaclassifiers for Text Learning.  CMU-CS-06-121, Ph.D. Thesis, Computer Science Department, School of Computer Science, Carnegie Mellon University.  May 2006. See errata.
• Using Asymmetric Distributions to Improve Classifier Probability Estimates.  Presentation at the NIPS 2004 Workshop on Calibration and Probabilistic Prediction in Supervised Learning.
• Leveraging Locality for Text Classifier Combination.  Presentation at the IJCAI 2003 Doctoral Consortium.
• R. Hutchinson, P.N. Bennett, J. Carbonell, P. Jansen, R. Brown (2003).  Maximal Lattice Overlap in Example-Based Machine Translation.  CMU-CS-03-138, Computer Science Department, School of Computer Science, Carnegie Mellon University, June 2003.  (Also listed as CMU-LTI-03-174. Shares significant overlap in content with publication listing for MT Summit IX, 2003.)
• P.N. Bennett (2002).  Using Asymmetric Distributions to Improve Classifier Probabilities: A Comparison of New and Standard Parametric Methods.  CMU-CS-02-126, Computer Science Department, School of Computer Science, Carnegie Mellon University, April 2002.  See errata. (A revised version of this work appears in SIGIR 2003.)
• P.N. Bennett and A.D. Brown, eds. (2000).  The Guide to Living in Pittsburgh, 9th Edition, 299 pages, Pittsburgh, PA.  Dec/5, Inc.  2000.
• P.N. Bennett (2000).  Assessing the Calibration of Naive Bayes' Posterior Estimates.  CMU-CS-00-155, Computer Science Department, School of Computer Science, Carnegie Mellon University.  September 2000.
• P.N. Bennett (1998).  Text Categorization Through Probabilistic Learning: Applications to Recommender Systems. Undergraduate Honor Thesis, Department of Computer Sciences, University of Texas at Austin. May 1998. (Also appears as AI TR 98-270.)
   

Other Activities:

• Co-organizer and Chair of SIGIR 2008 Workshop on Beyond Binary Relevance: Preferences, Diversity, and Set-Level Judgments
• Reviewer for ECML 2008 Workshop on Preference Learning.
• Reviewer for ECIR 2008.
• Reviewer for ACL 2008.
• Reviewer for AAAI Doctoral Consortium 2008.
• Reviewer for TOIS.
• Reviewer for JMLR.
• Publication Chair of ICML, 2006.
• Reviewer for ACM SIGIR 2004-2008.
• Reviewer for HLT-NAACL 2007.
• Reviewer for the Journal of Artificial Intelligence Research.
• Reviewer for ACM Transactions on Internet Technology.
• Co-organizer IR Seminar Series, Summer 2004 - Summer 2005.