AN3: A Low-Cost, Circuit-Switched Datacenter Network

In this paper, we present AN3—a low-cost, circuit-switched datacenter network. AN3 replaces expensive IP switches with custom hardware that supports circuit-based switching efficiently and with low cost. AN3 is enabled by a new speculative transmission protocol that (1) enables rapid multiplexing of links to efficiently support many flows in a datacenter-scale computer, and (2) establishes setup and teardown of circuits within tens of microseconds—well below the TCP handshake delay. In simulations, AN3 achieves over 90% link utilization in synthetic and real-world applications. A significant portion of AN3 has been built using low-cost, commodity FPGAs—our cost analysis shows that AN3 reduces the amortized per-server cost of the network by 1.7-2.4X relative to a conventional datacenter network.