Internet Economics and Computational Advertising (IECA) Group

About    People    News    Awards    Projects    Publications    Links


The IECA group consists of an elite team of researchers who have strong expertise in information retrieval, machine learning, data mining, game theory, and micro-economics. In particular, the group is devoted to the following research directions: learning to rank for Web search and online advertising, statistical game theory, game-theoretic machine learning, mechanism (economics model) design for online business, click and conversion prediction for sponsored search, etc. This group has been widely recognized for its outstanding research on learning to rank. Researchers from this group have published several award-winning papers and tens of highly-cited papers at top conferences including SIGIR, WWW, KDD, ICML and NIPS, have given tutorials at WWW, SIGIR and KDD, and have served as chairs, editors, and committee members for prestigious conferences and journals in the related research area. This group has become a key partner of Microsoft Advertising, and has contributed tens of granted patents and several core technologies to its products.


  • NEW! [2014-06] Tie-Yan Liu serves as one of Local co-chairs of ICML 2014, which will be held in Beijing, China. More details will be available later.
  • [2014] Tie-Yan Liu serves as one of PC co-chairs of WINE 2014, which will be held in Beijing, China. More details will be available later.
  • [2013-12] Tie-Yan Liu serves as one of PC co-chairs of AIRS 2013.
  • [2013-10] Jiang Bian serves as one of Local co-chairs of CIKM 2013.
  • NEW! [2013-09] Tao Qin's paper "Estimation Bias in Multi-Armed Bandit Algorithms for Search Advertising" is accepted by NIPS 2013.
  • NEW! [2013-09] Tao Qin's paper "Optimal Allocation for Chunked-Reward Advertising" is accepted by WINE 2013.
  • Hiring: We are hiring at all levels (from fresh graduates to experienced researchers)! If your major is machine learning, information retrieval, or algorithmic game theory, and micro-economics, and you have the passion to change the world, please send your resume to


  • MSRA FY12 Research Breakthrough Award (IECA Group)
  • SIGIR 2008 Best Student Paper Award (Bin Gao, Tie-Yan Liu, etc.)


Business Models for Internet Economies

The Internet has become a key part of the economic system. We are working on the design of advanced business models (mechanisms) for Internet economies. We are working on a number of new models for online businesses (such as online advertising, crowd-sourcing, and electronic commerce) and for offline businesses (such as software and living services) in the Internet era.

Statistical Game Theory

Games in many Internet businesses (e.g., online advertising) have very different properties from classical games: fast-paced, large-scale, complicated, dynamic, heterogeneous. In this case, the basic assumptions of classical game theory (i.e., players' full understanding of the game and opponents, and their rational decisions) do not hold. We target at refining the classical game theory by avoiding those unreasonable assumptions and leveraging large-scale empirical data to understand and describe how people play games.

Game-theoretic Machine Learning

Modern internet applications usually involve human. So it is no longer appropriate to assume the data generated in these applications to be static and i.i.d. Instead, it would be more reasonable to assume that the data is generated by human’s strategic behaviors in a game. This poses great challenges to classical machine learning algorithms and theories. We are working on a new machine learning framework to tackle this problem.

Learning to Rank for Search and Advertising

Ranking is a critical task in both search and advertising. We are working on how to use machine learning technologies to solve this task. We have proposed a listwise approach to ranking, and developed several effective learning-to-rank algorithms, such as ListNet, ListMLE, CCRF, Relational Ranking SVM. We have also made fundamental progress in statistical learning theory for ranking, by proving a two-layer generalization theory for ranking and revealing the strong connections between loss functions in learning-to-rank algorithms and widely-used ranking measures.

Computational Advertising

Online advertising is the major revenue source for many Internet companies. We are working on several key components in online advertising, including effective ad selection, ad relevance computation, ad click / conversion prediction, and automatic ad optimization. We are working together with Microsoft Advertising, and have transferred a couple of technologies to its products.


Selected Publications

Internet Economics and Computational Advertising
  1. Weihao Kong, Jian Li, Tao Qin, and Tie-Yan Liu, Optimal Allocation for Chunked-Reward Advertising, WINE 2013.
  2. Min Xu, Tao Qin, and Tie-Yan Liu, Estimation Bias in Multi-Armed Bandit Algorithms for Search Advertising, NIPS 2013.
  3. Di He, Wei Chen, Liwei Wang, and Tie-Yan Liu, Online Learning for Auction Mechanism in Bandit Setting, Decision Support Systems, 2013.
  4. Di He, Wei Chen, Liwei Wang, and Tie-Yan Liu, Probablistic Broad Match for Multi-Keyword Sponsored Search Auctions, EC AdAuction Workshop 2013.
  5. Taifeng Wang, Jiang Bian, Shusen Liu, Yuyu Zhang, and Tie-Yan Liu, Psychological Advertising: Exploring Consumer Psychology for Click Prediction in Sponsored Search, KDD 2013.
  6. Di He, Wei Chen, Liwei Wang, Tie-Yan Liu, A Game-theoretic Machine Learning Approach for Revenue Maximization in Sponsored Search, IJCAI 2013.
  7. Wenkui Ding, Tao Qin, and Tie-Yan Liu, Multi-Armed Bandit with Budget Constraint and Variable Costs, AAAI 2013.
  8. Haifeng Xu, Diyi Yang, Bin Gao and Tie-Yan Liu, Predicting Advertiser Bidding Behaviors in Sponsored Search by Rationality Modeling, WWW 2013.
  9. Lei Yao, Wei Chen and Tie-Yan Liu, Convergence Analysis for Weighted Joint Strategy Fictitious Play in Generalized Second Price Auction, WINE 2012.
  10. Weinan Zhang, Ying Zhang, Bin Gao, Yong Yu, Xiaojie Yuan, and Tie-Yan Liu, Joint optimization of bid and budget allocation in sponsored search, KDD 2012.
  11. Konstantin Salomatin, Tie-Yan Liu, and Yiming Yang, A Unified Optimization Framework for Auction and Guaranteed Delivery in Online Advertising, CIKM 2012.
  12. Chenyan Xiong, Taifeng Wang, Wenkui Ding, Yidong Shen, Tie-Yan Liu. Relational Click Prediction for Sponsored Search , WSDM 2012.
  13. Sungchul Kim, Tao Qin, Hwanjo Yu and Tie-Yan Liu, An Advertiser-Centric Approach to Understand User Click Behavior in Sponsored Search, CIKM 2011.
Learning to Rank
  1. Tie-Yan Liu. Learning to Rank for Information Retrieval, Springer, 2011.
    Yining Wang, Liwei Wang, Yuanzhi Li, Di He, Wei Chen, and Tie-Yan Liu, A Theoretical Analysis of NDCG Type Ranking Measures, COLT 2013.
  2. Yanyan Lan, Jiafeng Guo, Xueqi Cheng, Tie-Yan Liu, Statistical Consistency of Ranking Methods in A Rank-Differentiable Probability Space. NIPS 2012.
  3. Tao Qin, Xiubo Geng, and Tie-Yan Liu, A New Probabilistic Model for Rank Aggregation, NIPS 2010.
  4. Wei Chen, Tie-Yan Liu, Zhiming Ma, Two-Layer Generalization Analysis for Ranking Using Rademacher Average, NIPS 2010.
  5. Fen Xia, Tie-Yan Liu, Hang Li, Statistical Consistency of Top-k Ranking, NIPS 2009.
  6. Wei Chen, Tie-Yan Liu, Yanyan Lan, Zhiming Ma, Hang Li, Ranking Measures and Loss Functions in Learning to Rank, NIPS 2009.
  7. Tao Qin, Tie-Yan Liu, Xudong Zhang, and Hang Li. Global Ranking Using Continuous Conditional Random Fields, NIPS 2008.
  8. Yanyan Lan, Tie-Yan Liu, Zhiming Ma, and Hang Li. Generalization Analysis of Listwise Learning to Rank Algorithms, ICML 2009.
  9. Fen Xia, Tie-Yan Liu, Jue Wang, Wensheng Zhang, and Hang Li. Listwise Approach to Learning to Rank: Theorem and Algorithm, ICML 2008.
  10. Yanyan Lan, Tie-Yan Liu, Tao Qin, Zhiming Ma, and Hang Li. Query-level Stability and Generalization in Learning to Rank, ICML 2008.
  11. Zhe Cao, Tao Qin, Tie-Yan Liu, Ming-Feng Tsai, and Hang Li. Learning to Rank: From Pairwise Approach to Listwise Approach. ICML 2007.
  12. Olivier Chapelle, Yi Chang, and Tie-Yan Liu, Future research directions on learning to rank, Proceeding track, Journal of Machine Learning Research, 2011.
  13. Xiubo Geng, Tie-Yan Liu, Tao Qin, Andrew Arnold, Hang Li, and Heung-Yeung Shum. Query-dependent Ranking using K-Nearest Neighbor, SIGIR 2008.
  14. Jun Xu, Tie-Yan Liu, Min Lu, Hang Li, and Wei-Ying Ma. Directly Optimizing IR Evaluation Measures in Learning to Rank, SIGIR 2008.
  15. Tao Qin, Tie-Yan Liu, Jun Xu, and Hang Li. Making LETOR More Useful and Reliable, LR4IR 2008, in conjunction with SIGIR 2008.
  16. Xiubo Geng, Tie-Yan Liu, Tao Qin, and Hang Li. Feature Selection for Ranking, SIGIR 2007.
  17. Mingfeng Tsai, Tie-Yan Liu, Tao Qin, Hsin-Hsi Chen, and Wei-Ying Ma. FRank: A Ranking Method with Fidelity Loss, SIGIR 2007.
  18. Tao Qin, Tie-Yan Liu, Wei Lai, Xu-Dong Zhang, De-Sheng Wang, and Hang Li. Ranking with Multiple Hyperplanes, SIGIR 2007.
  19. Tie-Yan Liu, Jun Xu, Tao Qin, Wenying Xiong, and Hang Li. LETOR: Benchmark dataset for research on learning to rank for information retrieval, LR4IR 2007, in conjunction with SIGIR 2007.
  20. Yunbo Cao, Jun Xu, Tie-Yan Liu, Hang Li, Yalou Huang and Hsiao-Wuen Hon. Adapting Ranking SVM to Document Retrieval, SIGIR 2006.
  21. Tao Qin, Tie-Yan Liu, Xu-Dong Zhang, De-Sheng Wang, Wen-Ying Xiong, and Hang Li. Learning to Rank Relational Objects and Its Application to Web Search, WWW 2008.
  22. Yuting Liu, Tie-Yan Liu, Tao Qin, Zhi-Ming Ma, and Hang Li. Supervised Rank Aggregation, WWW 2007.
  23. Jiang Bian, Tie-Yan Liu, Tao Qin, and Hongyuan Zha, Ranking with query-dependent loss for web search. WSDM 2010.
  24. Yin He and Tie-Yan Liu, Tendency Correlation Analysis for Direct Optimization of Evaluation Measures in Information Retrieval, Information Retrieval Journal, 2010.
  25. Tao Qin, Tie-Yan Liu, and Hang Li, A General Approximation Framework for Direct Optimization of Information Retrieval Measures, Information Retrieval Journal, 2009.
  26. Xiubo Geng, Tao Qin, Xueqi Cheng, Tie-Yan Liu, A Noise-Tolerant Graphical Model for Ranking, Information Processing and Management, 2011.
  27. Xiubo Geng, Tie-Yan Liu, Tao Qin, Xueqi Cheng, Hang Li, Selecting Optimal Training Data for Learning to Rank, Information Processing and Management, 2011.
  28. Tie-Yan Liu. Learning to Rank for Information Retrieval, Foundation and Trends on Information Retrieval, Now Publishers, 2009.
Web Search and Data Mining
  1. Bin Gao, Tie-Yan Liu, Taifeng Wang, Wei Wei, and Hang Li, Semi-supervised graph ranking with rich meta data, KDD 2011.
  2. Bin Gao, Tie-Yan Liu, Yuting Liu, Taifeng Wang, Zhiming Ma, and Hang Li, Page Importance Computation based on Markov Processes, Information Retrieval, 2011.
  3. Zhicong Cheng, Bin Gao, Congkai Sun, Yanbing Jiang, and Tie-Yan Liu. Let Web Spammers Expose Themselves, WSDM 2011.
  4. Zhicong Cheng, Bin Gao, and Tie-Yan Liu, Actively Predicting Diverse Search Intent from User Browsing Behaviors, WWW 2010.
  5. Yuting Liu, Tie-Yan Liu, Zhiming Ma, and Hang Li. A Framework to Compute Page Importance based on User Behaviors, Information Retrieval Journal, 2009.
  6. Yuting Liu, Bin Gao, Tie-Yan Liu, Ying Zhang, Zhiming Ma, Shuyuan He, and Hang Li. BrowseRank: Letting Web Users Vote for Page Importance, SIGIR 2008. [SIGIR Best Student Paper Award]
  7. Guang Feng, Tie-Yan Liu, Ying Wang, Ting Bao, Zhiming Ma, Xu-Dong Zhang, and Wei-Ying Ma. AggregateRank: Bringing Order to Websites, SIGIR 2006.
  8. Tao Qin, Tie-Yan Liu, Xu-Dong Zhang, De-Sheng Wang, Zheng Chen, and Wei-Ying Ma. A Study on Relevance Propagation for Web Search, SIGIR 2005.
  9. Qiankun Zhao, Chuhong Hoi, Tie-Yan Liu, Sourav Bhowmick, Michael Lyu, and Wei-Ying Ma. Time-Dependent Semantic Similarity Measure of Queries Using Historical Click-Through Data, WWW 2006.
  10. Qiankun Zhao, Tie-Yan Liu, Sourav Bhowmick, and Wei-Ying Ma. Event Detection from Evolution of Click-through Data, KDD 2006.
  11. Bin Gao, Tie-Yan Liu, Xin Zheng, Qian-Sheng Cheng, and Wei-Ying Ma. Consistent Bipartite Graph Co-Partitioning for Star-Structured High-Order Heterogeneous Data Co-Clustering, KDD 2005.
  12. Ying Bao, Guang Feng*, Tie-Yan Liu, Zhiming Ma, and Ying Wang. Ranking Websites: A Probabilistic View, Internet Mathematics, 2007.
  13. Bin Gao, Tie-Yan Liu, Guang Feng, Tao Qin, Qian-Sheng Cheng, and Wei-Ying Ma. Hierarchical Taxonomy Preparation for Text Categorization Using Consistent Bipartite Spectral Graph Co-partitioning, IEEE Transactions on Knowledge and Data Engineering (TKDE), 2005.
  14. Tie-Yan Liu, Yiming Yang, Hao Wan, Hua-Jun Zeng, Zheng Chen, and Wei-Ying Ma. Support Vector Machines Classification with Very Large Scale Taxonomy, SIGKDD Explorations, 2005.
Multimedia Content Analysis
  1. Tie-Yan Liu, Kwok-Tung Lo, Xu-Dong Zhang, and Jian Feng. A New Cut Detection Algorithm with Constant False-Alarm Ratio for Video Segmentation, Journal of Visual Communications and Image Representation, 2004. [Most Cited Paper Award]
  2. Tie-Yan Liu, Xu-Dong Zhang, Jian Feng, and Kwok-Tung Lo. Shot Reconstruction Degree: a Novel Criterion for Key Frame Selection, Pattern Recognition Letters, 2004.
  3. Tie-Yan Liu, Kwok-Tung Lo, Jian Feng, and Xu-Dong Zhang. Frame Interpolation Scheme Using Inertia Motion Prediction. Signal Processing: Image Communication, 2003.
  4. Tie-Yan Liu, Jian Feng, Xu-Dong Zhang, and Kwok-Tung Lo. Inertia-based Cut Detection and Its Integration with Video Coder. IEE Proceedings on Vision, Image and Signal Processing, 2003.

Group Members
Tie-Yan Liu
Tie-Yan Liu

Bin Gao
Bin Gao

Taifeng Wang
Taifeng Wang

Tao Qin
Tao Qin

Wei Chen
Wei Chen

Jiang Bian
Jiang Bian

Feidiao Yang
Feidiao Yang

Weidong Ma
Weidong Ma


Past Visitors

  • Bo An (Assistant Professor, Nanyang Technological Univ., Singapore)
  • Pingzhong Tang (Assistant Professor, Tsinghua Univ., China)
  • Long Tran-Thanh (Research Staff, Univ. of Southampton, UK)
  • Lirong Xia (Assistant Professor, RPI, US)