Scheduling Interactive Services with Deadline and Partial Execution
This project explores a new scheduling problem for a class of interactive services in which requests are time bounded and lower result quality can be traded for shorter execution time. These applications include web search engines, finance servers, and other interactive, on-line services. We developed an efficient scheduling algorithm, Zeta, which allocates processor time among service requests to maximize the quality and minimize the variance of the response.
Zeta exploits the concavity of the request quality profile to distribute processing time among outstanding requests. By executing some requests partially (and obtaining much or most benefit of a full execution), Zeta frees resources for other requests, which might have timed out and produced no results. Compared to scheduling algorithms that consider only deadline or quality profile information, Zeta improves overall response quality and reduces response quality variance, yielding significant improvement in the high-percentile response quality.
We implemented Zeta in the Microsoft Bing web search engine and evaluated its performance in a production environment with realistic workloads. Measurements show that at the same response quality and latency as the production system, Zeta increases system capacity by 29% by improving both average and high percentile response quality. We also implemented Zeta in a finance server that computes option prices. In this application, Zeta improves average response quality by 28% and the 99-percentile quality by 80%. Using a simulation, we also compared Zeta to the offline optimal schedule and other scheduling algorithms. Although Zeta is only close to optimal, it provides better performance than prior algorithms under a wide variety of operating conditions.
- Yuxiong He, Sameh Elnikety, James Larus, and Chenyu Yan, Zeta: Scheduling Interactive Services with Partial Execution, in SoCC, SOCC '12 Proceedings of the 3rd ACM Symposium on Cloud Computing, October 2012.
- Yuxiong He, Zihao Ye, Qiang Fu, and Sameh Elnikety, Budget-based Control for Interactive Services with Adaptive Execution, in ICAC, International Conference on Autonomic Computing , September 2012.