Introduction to Parallel Algorithms and Architectures: Arrays, Trees, Hypercubes

This seminal work presents the only comprehensive integration of significant topics in computer architecture and parallel algorithms. The text is written for designers, programmers, and engineers who need to understand these issues at a fundamental level in order to utilize the full power afforded by parallel computation. It will also be an important resource for students and researchers. Writing for an advanced general audience, the author assumes few prerequisites while developing an elegant narrative of fundamental issues in parallel computation and applied algorithm design. The text is organized broadly by classes of network architectures and secondly by application domain. Emphasis is placed on developing a working knowledge of the essential paradigms of parallel computation, with techniques that are largely architecture independent. Basing the discussions on dominant communication networks that are the architectural basis of most parallel computing, the author describes the capabilities and limitations of the network architectures, as well as their use in solving specific algorithmic problems. This organization draws upon the reader's intuition by beginning with the most elementary and natural implementations. Starting with the simplest connection pattern of all-the linear array-and demonstrating its power with a number of illustrative applications, each subsequent algorithm is introduced using the simplest network on which it can be implemented. Throughout, the discussions consider the practical restrictions on hardware, with examples drawn from real implementations on commercially available machines. The thorough discussions and illuminating exercises offer significant insights about the subtleties that are the essence of parallel computing. The result for readers will be an in-depth understanding of parallel algorithms and architecture not available from any other source.