Can Parallel Computing Finally Impact Mainstream Computing?

The idea of upgrading performance and utility of computer systems by incorporating parallel processing has been around since at least the 1940s. A significant investment in parallel processing in the 1980s and 1990s has led to an abundance of parallel architectures that due to technical constraints at the time had to rely on multi-chip multi-processing. Unfortunately, their impact on mainstream computing was quite limited. “Tribal lore” suggests the following reason: while programming for parallelism tends to be easy, turning this parallelism into the “coarse-grained” type, needed to optimize performance for multi-chip multi-processing (with their high coordination overhead), has been quite hard. The mainstream computing paradigm has always relied on serial code. However, commercially successful processors are entering a second decade of near stagnation in the maximum number of instructions they can issue towards a single computational task in one clock. Alas, the multi-decade reliance on advancing the clock is also coming to an end.

The PRAM-On-Chip research project at UMD is reviewed. The project is guided by a fact, an old insight, a recent one and a premise. The fact: Billion transistor chips are here, up from less than 30,000 circa 1980. The older insight: Using a very simple parallel computation model, the parallel random access model (PRAM), the parallel algorithms research community succeeded in developing a general-purpose theory of parallel algorithms that was much richer than any competing approach and is second in magnitude only to serial algorithmics. However, since it did not offer an effective abstraction for multi-chip multi-processors this elegant algorithmic theory remained in the ivory towers of theorists.

The PRAM-On-Chip insight: The Billion transistor chip era allows for the first time low-overhead on-chip multi-processors so that the PRAM abstraction becomes effective. The premise: Were the architecture component of PRAM-On-Chip feasible in the 1990s, its parallel programming component would become the mainstream standard. In 1988-90 standard algorithms textbooks chose to include significant PRAM chapters (some still have them). Arguably nothing could stand in the way of teaching them to every student at every computer science program.

Programming for concurrency/parallelism is quickly becoming an integral part of mainstream computing. Yet, industry and academia leaders in system software and general-purpose application software have maintained a passive posture: their attention tends to focus too much on getting the best performance out of architectures that originated from a hardware centric approach, or very specific applications, and too little on trying to impact the evolving generation of multi-core and/or multi-threaded general-purpose architectures. We argue, perhaps provocatively, that: (i) limiting programming for parallelism to fit hardware-centric architectures imports the epidemic of programmability problems that has plagued parallel computing into mainstream computing, (ii) it is only a matter of time until the industry will seek convergence based on parallel programmability: the difference in the bottom line between a successful and a less successful strategy on parallel programmability will be too big to ignore, (iii) a more assertive position by such leaders is necessary, and (iv) the PRAM-On-Chip approach offers a more balanced way that avoids these problems.

URL: http://www.umiacs.umd.edu/~vishkin/XMT

Speaker Details

Uzi Vishkin is a permanent member of the University of Maryland Institute for Advanced Computer Studies, and Professor of Electrical and Computer Engineering. He was Professor of Computer Science at the Technion in Israel as well as at Tel Aviv University, Israel, where he was also Chair of the Computer Science Department. Prior to that he was with Courant Institute, New York University, and with IBM T.J. Watson Research. He is Fellow of the ACM.

Date:
Speakers:
Uzi Vishkin
Affiliation:
University of Maryland Institute for Advanced Computer Studies
    • Portrait of Jeff Running

      Jeff Running