2013

• Emma Brunskill and Lihong Li, Sample complexity of transfer reinforcement learning, in Proceedings of the Twenty-Nineth Conference on Uncertainty in Artificial Intelligence (UAI-13), Association for Uncertainty in Artificial Intelligence, July 2013

2012

• Olivier Chapelle and Lihong Li, An Empirical Evaluation of Thompson Sampling, in Advances in Neural Information Processing Systems 24 (NIPS-11), Neural Information Processing Systems Foundation, 2012
• Lihong Li, Wei Chu, John Langford, Taesup Moon, and Xuanhui Wang, An Unbiased Offline Evaluation of Contextual Bandit Algorithms based on Generalized Linear Models, in Journal of Machine Learning Research — Proceedings Track (ICML-2011 Workshop on Online Trading of Exploration and Exploitation), 2012
• Hongning Wang, Anlei Dong, Lihong Li, Yi Chang, and Evgeniy Gabrilovich, Joint Relevance and Freshness Learning from Clickthroughs for News Search, in Proceedings of the Twenty-First International Conference on World Wide Web (WWW-12), 2012
• Vidhya Navalpakkam, Ravi Kuma, Lihong Li, and Dandapani Sivakumar, Modeling Attention in Multi-Item Choice Tasks, in Proceedings of the Twentieth Conference on User Modeling, Adaptation, and Personalization (UMAP-2012), 2012
• Taesup Moon, Wei Chu, Lihong Li, Zhaohui Zheng, and Yi Chang, An Online Learning Framework for Refining Recency Search Results with User Click Feedback, in ACM Transactions on Information System, vol. 30, no. 4, 2012
• Miroslav Dudík, Dumitru Erhan, John Langford, and Lihong Li, Sample-efficient Nonstationary-policy Evaluation for Contextual Bandits, in Proceedings of the Twenty-Eighth Conference on Uncertainty in Artificial Intelligence (UAI-12), 2012
• Lihong Li, Sample Complexity Bounds of Exploration, in Reinforcement Learning: State of the Art, Springer Verlag, 2012

2011

• Lihong Li, Michael L. Littman, Thomas J. Walsh, and Alexander L. Strehl, Knows What It Knows: A Framework for Self-Aware Learning, in Machine Learning, vol. 82, no. 3, pp. 399–443, 2011
• Miroslav Dudík, John Langford, and Lihong Li, Doubly Robust Policy Evaluation and Learning, in Proceedings of the Twenty-Eighth International Conference on Machine Learning (ICML-11), 2011
• Alexander L. Strehl, John Langford, Lihong Li, and Sham M. Kakade, Learning from Logged Implicit Exploration Data, in Advances in Neural Information Processing Systems 23 (NIPS-10), 2011
• Martin Zinkevich, Markus Weimer, Alexander J. Smola, and Lihong Li, Parallelized Stochastic Gradient Descent, in Advances in Neural Information Processing Systems 23 (NIPS-10), 2011
• Lihong Li, Wei Chu, John Langford, and Xuanhui Wang, Unbiased Offline Evaluation of Contextual-bandit-based News Article Recommendation Algorithms, in Proceedings of the Fourth International Conference on Web Search and Web Data Mining (WSDM-11), 2011
• Wei Chu, Martin Zinkevich, Lihong Li, Achint Thomas, and Belle Tseng, Unbiased Online Active Learning in Data Streams, in Proceedings of the Seventeenth ACM SIGKDD Conference on Knowledge Discovery and Data Mining (KDD-11), 2011
• John Langford, Lihong Li, Preston McAfee, and Kishore Papineni, Cloud Control: Voluntary Admission Control for Intranet Traffic Management, in Information Systems and e-Business Management, 2011
• Wei Chu, Lihong Li, Lev Reyzin, and Robert E. Schapire, Contextual Bandits with Linear Payoff Functions, in Proceedings of the Fourteenth International Conference on Artificial Intelligence and Statistics (AISTATS-11), 2011
• Alina Beygelzimer, John Langford, Lihong Li, Lev Reyzin, and Robert E. Schapire, Contextual Bandit Algorithms with Supervised Learning Guarantees, in Proceedings of the Fourteenth International Conference on Artificial Intelligence and Statistics (AISTATS-11), 2011
• Deepak Agarwal, Lihong Li, and Alexander J. Smola, Linear-Time Estimators for Propensity Scores, in Proceedings of the Fourteenth International Conference on Artificial Intelligence and Statistics (AISTATS-11), 2011

2010

• Taesup Moon, Lihong Li, Wei Chu, Ciya Liao, Zhaohui Zheng, and Yi Chang, Online learning for recency search ranking using real-time user feedback, in Proceedings of the Nineteenth International Conference on Knowledge Management (CIKM-10), 2010
• Lihong Li, Wei Chu, John Langford, and Robert E. Schapire, A Contextual-Bandit Approach to Personalized News Article Recommendation, in Proceedings of the Nineteenth International Conference on World Wide Web (WWW-10), 2010
• Lihong Li and Michael L. Littman, Reducing Reinforcement Learning to KWIK Online Regression, in Annals of Mathematics and Artificial Intelligence, vol. 58, no. 3–4, pp. 217–237, 2010
• John Langford, Lihong Li, Yevgeniy Vorobeychik, and Jennifer Wortman, Maintaining Equilibria During Exploration in Sponsored Search Auctions, in Algorithmica, vol. 58, no. 4, pp. 990–1021, 2010

2009

• Lihong Li, Jason D. Williams, and Suhrid Balakrishnan, Reinforcement Learning for Spoken Dialog Management using Least-Squares Policy Iteration and Fast Feature Selection, in Proceedings of the Tenth Annual Conference of the International Speech Communication Association (INTERSPEECH-09), 2009
• Thomas J. Walsh, Ali Nouri, Lihong Li, and Michael L. Littman, Planning and Learning in Environments with Delayed Feedback, in Journal of Autonomous Agents and Multi-Agent Systems, vol. 18, no. 1, pp. 83–105, 2009
• Emma Brunskill, Bethany R. Leffler, Lihong Li, Michael L. Littman, and Nicholas Roy, Provably Efficient Learning with Typed Parametric Models, in Journal of Machine Learning Research, vol. 10, pp. 1955–1988, 2009
• Lihong Li, Michael L. Littman, and Christopher R. Mansley, Online Exploration in Least-Squares Policy Iteration, in Proceedings of the Eighteenth International Conference on Agents and Multiagent Systems (AAMAS-09), 2009
• Alexander L. Strehl, Lihong Li, and Michael L. Littman, Reinforcement Learning in Finite MDPs: PAC Analysis, in Journal of Machine Learning Research, vol. 10, pp. 2413–2444, 2009
• John Langford, Lihong Li, and Tong Zhang, Sparse Online Learning via Truncated Gradient, in Advances in Neural Information Processing Systems 21 (NIPS-08), 2009
• Carlos Diuk, Lihong Li, and Bethany R. Leffler, The Adaptive k-Meteorologists Problem and Its Application to Structure Discovery and Feature Selection in Reinforcement Learning, in Proceedings of the Twenty-Sixth International Conference on Machine Learning (ICML-09), 2009
• John Asmuth, Lihong Li, Michael L. Littman, Ali Nouri, and David Wingate, A Bayesian Sampling Approach to Exploration in Reinforcement Learning, in Proceedings of the Twenty-Fifth Conference on Uncertainty in Artificial Intelligence (UAI-09), 2009
• John Langford, Lihong Li, and Tong Zhang, Sparse Online Learning via Truncated Gradient, in Journal of Machine Learning Research, vol. 10, pp. 777–801, 2009
• Lihong Li, A Unifying Framework for Computational Reinforcement Learning Theory, New Brunswick, NJ, 2009

2008

• Lihong Li, A Worst-Case Comparison between Temporal Difference and Residual Gradient with Linear Function Approximation, in Proceedings of the Twenty-Fifth International Conference on Machine Learning (ICML-08), 2008
• Emma Brunskill, Bethany R. Leffler, Lihong Li, Michael L. Littman, and Nicholas Roy, CORL: A Continuous-State Offset-Dynamics Reinforcement Learner, in Proceedings of the Twenty-Fourth Conference on Uncertainty in Artificial Intelligence (UAI-08), 2008
• Lihong Li and Michael L. Littman, Efficient Value-Function Approximation via Online Linear Regression, in Proceedings of the Tenth International Symposium on Artificial Intelligence and Mathematics (ISAIM-08), 2008
• Lihong Li, Michael L. Littman, and Thomas J. Walsh, Knows What It Knows: A Framework for Self-Aware Learning, in Proceedings of the Twenty-Fifth International Conference on Machine Learning (ICML-08), 2008
• Ronald Parr, Lihong Li, Gavin Taylor, Christopher Painter-Wakefield, and Michael L. Littman, An Analysis of Linear Models, Linear Value-Function Approximation, and Feature Selection for Reinforcement Learning, in Proceedings of the Twenty-Fifth International Conference on Machine Learning (ICML-08), 2008

2007

• Thomas J. Walsh, Ali Nouri, Lihong Li, and Michael L. Littman, Planning and Learning in Environments with Delayed Feedback, in Proceedings of the Eighteenth European Conference on Machine Learning (ECML-07), 2007
• Lihong Li, Vadim Bulitko, and Russell Greiner, Focus of Attention in Reinforcement Learning, in Journal of Universal Computer Science, 2007
• Ronald Parr, Christopher Painter-Wakefield, Lihong Li, and Michael L. Littman, Analyzing Feature Generation for Value-Function Approximation, in Proceedings of the Twenty-Fourth International Conference on Machine Learning (ICML-07), 2007
• Jennifer Wortman, Yevgeniy Vorobeychik, Lihong Li, and John Langford, Maintaining Equilibria During Exploration in Sponsored Search Auctions, in Proceedings of the Third International Workshop on Internet and Network Economics (WINE-07), 2007

2006

• Lihong Li, Thomas J. Walsh, and Michael L. Littman, Towards a Unified Theory of State Abstraction for MDPs, in Proceedings of the Ninth International Symposium on Artificial Intelligence and Mathematics (ISAIM-06), 2006
• Alexander L. Strehl, Lihong Li, and Michael L. Littman, Incremental Model-based Learners with Formal Learning-Time Guarantees, in Proceedings of the Twenty-Second Conference on Uncertainty in Artificial Intelligence (UAI-06), 2006
• Alexander L. Strehl, Lihong Li, Eric Wiewiora, John Langford, and Michael L. Littman, PAC Model-Free Reinforcement Learning, in Proceedings of the Twenty-Third International Conference on Machine Learning (ICML-06), 2006

2005

• Lihong Li and Michael L. Littman, Lazy Approximation for Solving Continuous Finite-Horizon MDPs, in Proceedings of the Twentieth National Conference on Artificial Intelligence (AAAI-05), 2005

2004

• Lihong Li, Vadim Bulitko, and Russell Greiner, Batch Reinforcement Learning with State Importance, in Proceedings of the Fifteenth European Conference on Machine Learning (ECML-04), 2004

2003

• Vadim Bulitko, Lihong Li, Russell Greiner, and Ilya Levner, Lookahead Pathologies for Single Agent Search, in Proceedings of the Eighteenth International Joint Conferences on Artificial Intelligence (IJCAI-03), 2003