Share on Facebook Tweet on Twitter Share on LinkedIn Share by email

Experience from an operational map-reduce cluster reveals that outliers significantly prolong job completion. Mantri culls outliers using cause- and resource-aware techniques. Its strategies include smart restart of outliers, network-aware placement of tasks and protecting outputs of valuable tasks. Deployment in Bing’s production cluster and extensive trace-driven simulation indicate that Mantri is 3.1x more effective than the existing state-of-the-art in improving job completion times.



Ganesh Ananthanarayana UC Berkeley (MSR Intern)
Srikanth Kandula  MSR 
Albert Greenberg  MSR 
Ion Stoica   UC Berkeley 
Yi Lu   MSR, now at UIUC
Bikas Saha Bing 
Edward Harris Bing 


Combating Stragglers in MapReduce Networks @ Brown IPP Symposium on Cloud Computing May, 2010

Combating Stragglers in MapReduce Networks @ MSR-Technology Advisory Board Meeting, July 2010