Ant Rowstron
Principal Researcher, Microsoft Research Cambridge
Rethinking the data center: CamCube and beyond
We have been looking at how to build a cluster to support data processing by combining idea from High Performance Computing, Distributed Systems and Networking. Key ideas we have explored is what happens if we can combine the network and compute, and then map computation (currently data analytics frameworks) on to the platform. We have a MapReduce implementation that runs using the same functions that you have in Hadoop/MapReduce, but it is able to run the combiner function in network (see the NSDI 2012 paper). The interconnect that we use is a 3D Torus.Recently, this work has more generally led us to think about design of clusters and individual servers for use in the data center. We have been exploring the design of clusters with asymmetric memory across the servers (the Jekyll and Hyde Cluster designed to support mixed workloads (e.g. a high-throughput OLTP-like workload combined with a data analytics workload that runs concurrently)).
This in turn has led us to think about the short term impact of increasing DRAM size per server. It is now financially reasonable (as well as technically feasible) to have 100's of GB of DRAM per server (e.g. 192GB is 12 x 16GB DIMMS = a couple of thousand dollars). Is single server memory scaling (scale up) better than scaling to a cluster (scale out)?
Conference papers:
Paolo Costa, Austin
Donnelly, Ant Rowstron, Greg O'Shea. "Camdoop: Exploiting
In-network Aggregation for Big Data Applications".
Proceedings NSDI, April, 2012
[
pdf
]
H. Abu-Libdeh, P. Costa, A. Rowstron, G.
O'Shea and A. Donnelly.
"Symbiotic routing in future data centers"
. Proceedings ACM Sigcomm, Aug 2010.
[
pdf
]
P. Costa, T. Zhan, A. Rowstron, G. O'Shea and S. Schubert.
"Why should we
integrate services, servers, and networking in a Data Center?"
Proceedings of WREN, August 2009.
[
pdf ]
Workshop paper describing early experience scaling up memory versus scaling out using a cluster:
News
Recently saw the SeaMicro product:
http://www.seamicro.com
Imagine Camdoop running on that!
Camdoop paper accepted for NSDI 2012.