WiDS: an Integrated Toolkit for Distributed System Development

  • Shiding Lin ,
  • Aimin Pan ,
  • Zheng Zhang ,
  • Rui Guo ,
  • Zhenyu Guo

HotOS |

Published by USENIX

Publication

Faced with a proliferation of distributed systems in research and production groups, we have devised the WiDS ecosystem of technologies to optimize the development and testing process for such systems. WiDS optimizes the process of developing the algorithm, testing its correctness in a debuggable environment, and testing its behavior at large scales in a distributed simulation. We have developed many distributed protocols and systems using WiDS, including a large-scale backup service that is robust enough to be deployed. We have also used WiDS to perform ultralarge scale (>1million instances) simulation for production protocol. In this paper, we describe the principle and design of WiDS, share the lessons that we learned, and iscuss ongoing researches that will further reduce programming and debugging difficulties of distributed systems.