Cloud data centers host diverse applications, mixing workloads that require

small predictable latency with others requiring large sustained throughput. In

this environment, today's state-of-the-art TCP protocol falls short. We present

measurements of a 6000 server production cluster and reveal impairments that

lead to high application latencies, rooted in TCP's demands on the limited

buffer space available in data center switches. For example, bandwidth hungry

``background'' flows build up queues at the switches, and thus impact the

performance of latency sensitive ``foreground'' traffic.

To address these problems, we propose DCTCP, a variant of TCP for data center

networks. DCTCP leverages Explicit Congestion Notification (ECN) in the network

to provide multi-bit feedback to a simple control mechanism implemented in the

host OS. We evaluate DCTCP at 1 and 10Gbps speeds, through benchmark

experiments and analysis. In the data center, operating with commodity, shallow

buffered switches, we find DCTCP delivers the same or better throughput than

TCP, while using 90\% less buffer space. Unlike TCP, DCTCP also provides high

burst tolerance and low latency for short flows. In handling workloads derived

from operational measurements, we found DCTCP enables the applications to handle

10X the current background traffic, without impacting foreground traffic.

Further, a 10X increase in foreground traffic does not cause any timeouts, thus

largely eliminating incast problems.

Publisher  Microsoft Research
© 2009 Microsoft Corporation. All rights reserved.

Details