Information technology (IT) has the potential to continue to dramatically transform how we work and live. One might expect that future IT advances will occur as a natural continuation of the stunning advances that IT has enabled over the last half-century, but reality is more sobering.
IT advances of the last half-century have depended critically on the rapid growth of single-processor performance—by a factor of 10,000 in just the last 2 decades—at ever-decreasing cost and with manageable increases in power consumption. That growth stemmed from increasing the number and speed of transistors on a processor chip by reducing their size and—with improvements in memory, storage, and networking capacities—resulted in ever more capable computer systems. It was important for widespread IT adoption that the phenomenal growth in performance was achieved while maintaining the sequential stored-program model that was developed for computers in the 1940s. Moreover, computer manufacturers worked to ensure that specific instruction set compatibility was maintained over generations of computer hardware—that is, a new computer could run new applications, and the existing applications would run faster. Thus, software did not have to be rewritten for each hardware generation, and so ambition and imagination were free to drive the creation of increasingly innovative, capable, and computationally intensive software, and this in turn inspired businesses, government, and the average consumer to buy successive generations of computer software and hardware. Software and hardware advances fed each other, creating a virtuous IT economic cycle.
Early in the 21st century, improvements in single-processor performance slowed, as measured in instructions executed per second, and such performance now improves at a very modest pace, if at all. This abrupt shift is due to fundamental limits in the power efficiency of complementary metal oxide semiconductor integrated circuits (used in virtually all computer chips today) and apparent limits in the efficiencies that can be exploited in single-processor architectures. Reductions in transistor size continue apace, and so more transistors can still be packed onto chips, albeit without the speedups seen in the past. As a result, the computer-hardware industry has commenced building chips with multiple processors. Current chips range from several complex processors to hundreds of simpler processors, and future generations will keep adding more. Unfortunately, that change in hardware requires a concomitant change in the software programming model. To use chip multiprocessors, applications must use a parallel programming model, which divides a program into parts that are then executed in parallel on distinct processors. However, much software today is written according to a sequential programming model, and applications written this way cannot easily be sped up by using parallel processors.
The only foreseeable way to continue advancing performance is to match parallel hardware with parallel software and ensure that the new software is portable across generations of parallel hardware. There has been genuine progress on the software front in specific fields, such as some scientific applications and commercial searching and transactional applications. Heroic programmers can exploit vast amounts of parallelism, domain-specific languages flourish, and powerful abstractions hide complexity. However, none of those developments comes close to the ubiquitous support for programming parallel hardware that is required to ensure that IT’s effect on society over the next two decades will be as stunning as it has been over the last half-century.
For those reasons, the Committee on Sustaining Growth in Computing Performance recommends that our nation place a much greater emphasis on IT and computer-science research and development focused on improvements and innovations in parallel processing, and on making the transition to computing centered on parallelism. The following should have high priority:
- Algorithms that can exploit parallel processing;
- New computing “stacks” (applications, programming languages, compilers, runtime/virtual machines, operating systems, and architectures) that execute parallel rather than sequential programs and that effectively manage software parallelism, hardware parallelism, power, memory, and other resources;
- Portable programming models that allow expert and typical programmers to express parallelism easily and allow software to be efficiently reused on multiple generations of evolving hardware;
- Parallel-computing architectures driven by applications, including enhancements of chip multiprocessors, conventional data-parallel architectures, application-specific architectures, and radically different architectures;
- Open interface standards for parallel programming systems that promote cooperation and innovation to accelerate the transition to practical parallel computing systems; and
- Engineering and computer-science educational programs that incorporate an increased emphasis on parallelism and use a variety of methods and approaches to better prepare students for the types of computing resources that they will encounter in their careers.
Although all of those areas are important, fundamental power and energy constraints mean that even the best efforts might not yield a complete solution. Parallel computing systems will grow in performance over the long term only if they can become more power-efficient. Therefore, in addition to a focus on parallel processing, we need research and development on much more power-efficient computing systems at all levels of technology, including devices, hardware architecture, and software systems.