Database management systems provide functionality that is central to developing business applications. Therefore, database management systems are increasingly being used as an important component in applications. Yet, the problem of tuning database management systems for achieving required performance is significant, and results in high total cost of ownership (TCO). The goal of our research in the AutoAdmin project is to make database systems self-tuning and self-administering. We achieve this by enabling databases to track the usage of their systems and to gracefully adapt to application requirements. Thus, instead of applications having to track and tune databases, databases actively auto-tunes itself to be responsive to application needs. Our research has led to novel self-tuning components being included in Microsoft SQL Server.

This is a long-term project. In the short term, the focus is on automating physical database design. Physical database design is the problem of identifying the physical design structures such as indexes, materialized views and partitioning that are appropriate for the database and workload so as to optimize performance.

We have developed an Index Tuning Wizard that shipped as part of Microsoft SQL Server 7.0 and Microsoft SQL Server 2000, and the Database Engine Tuning Advisor (DTA) that shipped as part of Microsoft SQL Server 2005. Given a  workload consisting of a set of SQL statements, the Database Engine Tuning Advisor automatically recommends a set of indexes, materialized views and partitioning that are appropriate for that workload. A representative workload for the system can be gathered by logging activity on the server over a specified period of time. The user can specify constraints that must be met by the tool e.g., an upper bound on the storage space, or indexes that must be included. The tool also allows the user to perform a quantitative analysis of the impact of the proposed recommendation. For example, the database administrator can find the queries in the workload that are most affected by the proposed changes; or determine the fraction of queries where a particular index or materialized view is used. The tool also reports an expected improvement in performance if the proposed recommendations are accepted. If the user accepts the recommendations, the tool creates (and/or drops) indexes so that the recommended design is materialized. This work was done jointly with the SQL Server team. More details of the tool including its underlying algorithms are available in the VLDB'97 paper "An Efficient Cost-Driven Index Selection Tool for Microsoft SQL Server ", the VLDB'00 paper "Automated Selection of Materialized Views and Indexes for SQL Databases", the SIGMOD'04 paper "Integrating Vertical and Horizontal Partitioning into Automated Physical Database Design" and the VLDB'04 paper "Database Tuning Advisor for Microsoft SQL Server 2005". (see Publications below).

Sanjay Agrawal

Nicolas Bruno

Surajit Chaudhuri

Arnd Christian König

Vivek Narasayya

Ravishankar Ramamurthy

Manoj Syamala

The following papers are in pdf format. Click here to install Adobe Acrobat Reader.

Chaudhuri S. Ramamurthy R. and Narasayya V. , Diagnosing Estimation Errors in Page Counts using Execution Feedback Proceedings of the 2008 ICDE Conference. pdf version

Chaudhuri S., and Narasayya V., Self-Tuning Database Systems: A Decade of Progress. Proceedings of the 33rd International Conference on Very Large Databases (VLDB07), Vienna, Austria, 2007. (Invited paper for 10-year Best Paper Award). pdf version 

Chaudhuri S., Narasayya V., and Syamala M. , Bridging the Application and DBMS Profiling Divide for Database Application Developers. Proceedings of the 33rd International Conference on Very Large Databases (VLDB07), Vienna, Austria, 2007. pdf version 

Chaudhuri S., Kaushik R., Pol, A. and Ramamurthy R., Stop-and-Restart Style Query Execution for Long Running Decision Support Queries. Proceedings of the 33rd International Conference on Very Large Databases (VLDB07), Vienna, Austria, 2007.. pdf version 

Bruno N. and Chaudhuri S. , An Online Approach to Physical Design Tuning. Proceedings of the 2007 ICDE Conference. pdf version

Bruno N. and Chaudhuri S. , To Tune or not to Tune? A Lightweight Physical Design Alerter. Proceedings of the 2006 VLDB Conference. pdf version

Agrawal S., , Chu, E. and Narasayya V., Automated Physical Design Tuning: Workload as a Sequence. Proceedings of the ACM SIGMOD, Chicago, USA, 2006. pdf version

Kabra, G. , Ramamurthy R. and Sudarshan, S. Redundancy and Information Leakage in Fine Grained Access Control. Proceedings of the ACM SIGMOD, Chicago, USA, 2006. pdf version

König, A and Nabar, S., Scalable Exploration of Physical Database Design. Proceedings of 22th International Conference on Data Engineering, Atlanta, USA, 2006 pdf version

Bruno N. and Chaudhuri S. , Physical Design Refinement. The Merge-Reduce Approach. Proceedings of the 2006 EDBT Conference. pdf version

Babcock, B. and Chaudhuri S., Towards A Robust Query Optimizer: A Principled And Practical Approach. Proceedings of the ACM SIGMOD, Baltimore, USA, 2005. pdf version

Chaudhuri S., Kaushik, R, and Ramamurthy R., Can We Trust Progress Estimators For SQL Queries? Proceedings of the ACM SIGMOD, Baltimore, USA, 2005. pdf version

Chen Z. and Narasayya V. , Efficient Computation of Multiple Group-By Queries. Proceedings of the ACM SIGMOD, Baltimore, USA, 2005. pdf version

Bruno N. and Chaudhuri S. , Automatic Physical Design Tuning: A Relaxation Based Approach. Proceedings of the ACM SIGMOD, Baltimore, USA, 2005. pdf version

Agrawal S., Chaudhuri S., Kollar L., Marathe A., Narasayya V., and Syamala M., Database Tuning Advisor for Microsoft SQL Server 2005. Proceedings of the 30th International Conference on Very Large Databases (VLDB04), Toronto, Canada, 2004. pdf version 

Chaudhuri S., Narasayya V., and Ramamurthy R., Estimating Progress of Execution for SQL Queries. Proceedings of the ACM SIGMOD, Paris, France, 2004.  pdf version

Agrawal S., Narasayya V., and Yang, Berverly, Integrating Vertical and Horizontal Partitioning into Automated Physical Database Design. Proceedings of the ACM SIGMOD, Paris, France, 2004. pdf version

Bruno N. and Chaudhuri S. , Conditional Selectivity for Statistics on Query Expressions. Proceedings of the ACM SIGMOD , Paris, France, 2004.pdf version

Chaudhuri S., König, A., and Narasayya V. , SQLCM: A Continuous Monitoring Framework for Relational Database Engines. Proceedings of 20th International Conference on Data Engineering, Boston, USA, 2004. pdf version 

Chaudhuri S., Ganesan P.,, and Narasayya V. , Primitives for Workload Summarization and Implications for SQL Proceedings of the 29th International Conference on Very Large Databases (VLDB03), Berlin, Germany, 2003. pdf version 

Agrawal S., Chaudhuri S., Abhinandan Das, and Narasayya V. , Automating Layout of Relational Databases. Proceedings of 19th International Conference on Data Engineering, Bangalore, India, 2003. pdf version  

Agrawal S., Chaudhuri S., Kollar L., and Narasayya V. Index Tuning Wizard for Microsoft SQL Server 2000. White paper

Chaudhuri S. and Weikum G., Rethinking Database System Architecture: Towards a Self-tuning, RISC-style Database System . Proceedings of the 26th International Conference on Very Large Databases (VLDB00), Cairo, Egypt, 2000, pp. 1-10, 2000. pdf version

Agrawal S., Chaudhuri S. and Narasayya V., Automated Selection of Materialized Views and Indexes for SQL Databases. Proceedings of the 26th International Conference on Very Large Databases (VLDB00), Cairo, Egypt, 2000, pp. 496-505, 2000. pdf version

Chaudhuri S. and Narasayya V. , Automating Statistics Management for Query Optimizers. Proceedings of 16th International Conference on Data Engineering, San Diego, USA 2000. pdf version of Conference paper 
IEEE Transactions on Knowledge and Data Engineering (TKDE) Vol 13. No 1. January/February 2001. pdf version of Journal paper 

Aboulnaga, A. and Chaudhuri, S., Self-Tuning Histograms: Building Histograms Without Looking at Data. Proceedings of ACM SIGMOD, Philadelphia, 1999. pdf version

If you have questions about this project, please contact Surajit Chaudhuri (surajitc@microsoft.com).