File and Network I/O using Win32 and .NET API’s on Windows XP™
Jim Gray, Peter Kukol
In 2005 we update the Windows I/O studies in light of changes over the last 5 years. The modern studies use SATA disks, and use both native and unmanaged C++, and managed C# code (running on top of the .NET Runtime).
The basic conclusions of this 2005 study are as follows:
The updated diagram at right summarizes some of these speeds. It is based on the original Riedel 1997 study (below).
Here are the supporting documents:
· A report describing the SATA disk I/O measurements: SataDiskIO.doc (875 KB MS Word file).
· A document explaining the best approaches to doing file I/O using the .NET framework: Sequential File Programming Patterns and Performance with .NET (MS Research Technical Report MSR-TR-2004-136).
· A report discussing some of the findings of our trans-continental GB/sec disk transfer effort: Sequential Disk IO Tests for GBps Land Speed Record (MS Research Technical Report MSR-TR-2004-62).
· A collection of programs (each includes a Visual Studio 2003 project with source code and an x86 executable) to measure disk, network and memory speed (plus a few helper utilities):
1. The DiskSpd program to measure disk bandwidth (DiskSpd.zip).
2. The NetSpd program to measure network bandwidth (NetSpd.zip).
3. The MemSpd program to measure memory bandwidth (MemSpd.zip).
4. The GenFile program to quickly create large disk files with randomized content (GenFile.zip).
5. The DumpFile program displays sections of arbitrarily large files in hex and/or ASCII (DumpFile.zip).
Study of Random and Sequential IO on Windows 2000™
Leonard Chung, Jim Gray, Bruce Worthington, Microsoft,
Robert Horst, 3Ware
In 1997, Riedel, van Ingen, and Gray did a study of sequential IO on Windows NT 4.0 (see 2nd half of this page). We repeated that study and generalized it to include random IO, IDE as well as SCSI disks, and also RAID and network (CIFS/SMB) IO. The basic conclusions of the study are:
The RAP/PAP diagram of Riedel et al is revised as show in the diagram at the right:
Here are the supporting documents:
· The SQLIO2 program as a Visual Studio project and an exe (sqlio2.zip).
· The CashFlush program used in our tests to clear the cache CacheFlush.zip (58 KB).
· The MemSpeed program used in our tests to measure CPU, memory, controller, and disk speed as MemSpeed.zip (43 KB zip file).
· The FileExtendSpeed program used to measure how quickly files can grow FileExtendSpeed.zip (50 KB).
· Spreadsheets containing all our measurements results.zip (5 MB).
1997 Study of Sequential IO on Windows NT 4.0™
High-speed sequential file access is important for bulk data operations typically found in utility, multimedia, data mining, and scientific applications. High-speed sequential IO is also important in the startup of interactive applications. Minimizing IO overhead and maximizing bandwidth frees power to process the data. The goals of this study were twofold:
The report shows that NTFS out-of-the box read and write performance is quite good, but overheads for small requests can be quite high. The best performance is achieved by using large requests, bypassing the file system cache, spreading the data across many disks and controllers, and using deep-asynchronous requests.
PAP (Peak Advertised Performance) vs RAP (Real Achieved Performance)
The paper: (MSWord, 2MB)
The sample programs: (WinZip 19KB)
The program that generated the experimental data: (WinZip 66KB)
The talk Jim gave Dec 1 in Redmond (Gray_ECS_Disk.ppt 1.5MB)
The NetShow of the talk
Jim gave in
Note: In December 1999 Tadeusz Wellenger [email@example.com] reported a bug in the IO completion routine +Event logic. We fixed that and added a IO_CompletionPort example which is the "right" way to do asynchronous IO under NT and Windows 2000.