National Academies Press: OpenBook

Statistical Software Engineering (1996)

Chapter: Executive Summary

« Previous: Front Matter
Suggested Citation:"Executive Summary." National Research Council. 1996. Statistical Software Engineering. Washington, DC: The National Academies Press. doi: 10.17226/5018.
×

Executive Summary

Software, a critical core industry that is essential to U.S. interests in science, technology, and defense, is ubiquitous in today's society. Software coexists with hardware in our transportation, communication, financial, and medical systems. As these systems grow in size and complexity and our dependence on them increases, the need to ensure software reliability and safety, fault tolerance, and dependability becomes paramount. Building software is now viewed as an engineering discipline, software engineering, which aims to develop methodologies and procedures to control the whole software development process. Besides the issue of controlling and improving software quality, the issue of improving the productivity of the software development pro cess is also becoming important from the industrial perspective.

PURPOSE AND SCOPE OF THIS STUDY

Although statistical methods have a long history of contributing to improved practices in manufacturing and in traditional areas of science, technology, and medicine, they have up to now had little impact on software development processes. This report attempts to bridge the islands of knowledge and experience between statistics and software engineering by enunciating a new interdisciplinary field: statistical software engineering. It is hoped that the report will help seed the field of statistic al software engineering by indicating opportunities for statistical thinking to contribute to increased understanding of software and software production, and thereby enhance the quality and productivity of both.

This report is the result of a study by a panel convened by the Committee on Applied and Theoretical Statistics (CATS), a standing committee of the Board on Mathematical Sciences of the National Research Council, to identify challenges and opportunities in the development and implementation of software involving significant statistical content. In addition to pointing out the relevance of rigorous statistical and probabilistic techniques to pressing software engineering concerns, the panel outlines opp ortunities for further research in the statistical sciences and their applications to software engineering. The aim is to motivate new researchers from statistics and the mathematical sciences to tackle problems with relevance for software development, as well as to suggest a statistical approach to software engineering concerns that the panel hopes software engineers will find refreshing and stimulating. This report also touches on important issues in training and education for software engineers in the statistical sciences and for statisticians with an interest in software engineering.

Central to this report's theme, and essential to statistical software engineering, is the role of data: wherever data are used or can be generated in the software life cycle, statistical methods can be brought to bear for description, estimation, an d prediction. Nevertheless, the major obstacle to applying statistical methods to software engineering is the lack of consistent, high-quality data in the resource-allocation, design, review, implementation, and test stages of software development. Statisticians interested in conducting research in software engineering

Suggested Citation:"Executive Summary." National Research Council. 1996. Statistical Software Engineering. Washington, DC: The National Academies Press. doi: 10.17226/5018.
×

must play a leadership role in justifying that resources are needed to acquire and maintain high-quality and relevant data.

The panel conjectures that the use of adequate metrics and data of good quality is the primary differentiator between successful, productive software development organizations and those that are struggling. Although the single largest area of overlap between statistics and software engineering currently concerns software development and production, it is the panel's view that the largest contributions of statistics to software engineering will be those affecting the quality and productivity of front-en d processes, that is, processes that precede code generation. One of the biggest impacts that the statistical community can make in software engineering is to combine information across software engineering projects as a means of evaluating effects of technology, language, organization, and process.

CONTENTS OF THIS REPORT

Following an introductory opening chapter intended to familiarize readers with basic statistical software engineering concepts and concerns, a case study of the National Aeronautics and Space Administration (NASA) space shuttle flight control software is presented in Chapter 2 to illustrate some of the statistical issues in software engineering. Chapter 3 describes a well-known general software production model and associated statistical issues and approaches. A critique of some current applications of statistics and software engineering is presented in Chapter 4. Chapter 5 discusses a number of statistical challenges arising in software engineering, and the panel's closing summary and conclusions appear in Chapter 6.

STATISTICAL CHALLENGES

In comparison with other engineering disciplines, software engineering is still in the definition stage. Characteristics of established disciplines include having defined, tested, credible methodologies for practice, assessment, and predictability. Software engineering combines application domain knowledge, computer science, statistics, behavioral science, and human factors issues. Statistical challenges in software engineering discussed in this report include the following:

  • Generalizing particular statistical software engineering experimental results to other settings and projects,

  • Scaling up results obtained in academic studies to industrial settings,

  • Combining information across software engineering projects and studies,

  • Adopting exploratory data analysis and visualization techniques,

  • Educating the software engineering community regarding statistical approaches and data issues,

  • Developing methods of analysis to cope with qualitative variables,

Suggested Citation:"Executive Summary." National Research Council. 1996. Statistical Software Engineering. Washington, DC: The National Academies Press. doi: 10.17226/5018.
×
  • Providing models with the appropriate error distributions for software engineering applications, and

  • Enhancing accelerated life testing.

SUMMARY AND CONCLUSIONS

In the 1990s, complex hardware-based functionality is being replaced by more flexible, software-based functionality, and massive software systems containing millions of lines of code are being created by many programmers with different backgrounds, training, and skills.The challenge is to build huge, high-quality systems in a cost-effective manner. The panel expects this challenge to preoccupy the field of software engineering for the rest of the decade. Any set of methodologies that can help in this task will be invaluable. More importantly, the use of such methodologies will likely determine the competitive positions of organizations and nations involved in software production. What is needed is a detailed understanding by statisticians of the software engineering process, as well as an appreciation by software engineers of what statisticians can and cannot do.

Catalysts essential for this productive interaction between statisticians and software engineers, and some of the interdisciplinary research opportunities for software engineers and statisticians, include the following:

  • A model for statistical research in software engineering that is collaborative in nature. The ideal collaboration partners statisticians, software engineers, and a real software process or product. Barriers to academic reward and recognition barriers, as well as obstacles to the funding of cross-disciplinary research, can be expected to decrease over time; in the interim, industry can play a leadership role in nurturing collaborations between software engineers and statisticians and can reduce its own set of barriers (for instance, those related to proprietary and intellectual property interests).

  • A model for data collection and analysis that ensures the availability of high-quality data for statistical approaches to issues in software engineering. Careful attention to data issues ranging from definition of metrics to feed-back/-forward loops, including exploratory data analysis, statistical modeling, defect analysis, and so on, is essential if statistical methods are to have any appreciable impact on a given software project under study. For this reason it is crucial that the software industry take a lead position in research on statistical software engineering.

  • Attention to relevant issues in education. Enormous opportunities and many potential benefits are possible if the software engineering community learns about relevant statistical methods and if statisticians contribute to and cooperate in the education of future software engineers. Some relevant areas include:

Suggested Citation:"Executive Summary." National Research Council. 1996. Statistical Software Engineering. Washington, DC: The National Academies Press. doi: 10.17226/5018.
×
  • Designed experiments. Software engineering is inherently experimental, yet relatively few designed experiments have been conducted. Software engineering education programs must stress the desirability, where feasible, of validating new techniques using statistically valid designed experiments.

  • Exploratory data analysis. Exploratory data analysis methods are essentially ''model free," whereby the investigator hopes to be surprised by unexpected behavior rather than having thinking constrained to what is expected. Modeling. Recent advances in the statistical community in the past decade have effectively relaxed the linearity assumptions of nearly all classical techniques. There should be an emphasis on educational information exchange leading to more and wider use of these recently developed techniques.

  • Risk analysis. A paradigm for managing risk for the space shuttle program, discussed in Chapter 2 of this report, and the corresponding statistical methods can play a crucial role in identifying risk-prone parts of software systems and of combined hardware and software systems.

  • Attitude toward assumptions. Software engineers should be aware that violating assumptions is not as important as thoroughly understanding the violation's effects on conclusions. Statistics textbooks, courses, and consulting activities should convey the statistician's level of understanding about and perspective on the importance and implications of assumptions for statistical inference methods.

  • Visualization. Graphics is important in exploratory stages in helping to ascertain how complex a model the data ought to support; in the analysis stage, by which residuals are displayed to examine what the currently entertained model has failed to account for; and in the presentation stage, in which graphics can provide succinct and convincing summaries of the statistical analysis and the associated uncertainty. Visualization can help software engineers cope with, and understand, the huge quantities of data collected as part of the software development process.

  • Tools. It is important to identify good statistical computing tools for software engineers. An overview of statistical computing, languages, systems, and packages should be done that is focused specifically for the benefit of software engineers.

Suggested Citation:"Executive Summary." National Research Council. 1996. Statistical Software Engineering. Washington, DC: The National Academies Press. doi: 10.17226/5018.
×
Page 1
Suggested Citation:"Executive Summary." National Research Council. 1996. Statistical Software Engineering. Washington, DC: The National Academies Press. doi: 10.17226/5018.
×
Page 2
Suggested Citation:"Executive Summary." National Research Council. 1996. Statistical Software Engineering. Washington, DC: The National Academies Press. doi: 10.17226/5018.
×
Page 3
Suggested Citation:"Executive Summary." National Research Council. 1996. Statistical Software Engineering. Washington, DC: The National Academies Press. doi: 10.17226/5018.
×
Page 4
Next: Introduction »
Statistical Software Engineering Get This Book
×
Buy Paperback | $40.00 Buy Ebook | $31.99
MyNAP members save 10% online.
Login or Register to save!
Download Free PDF

This book identifies challenges and opportunities in the development and implementation of software that contain significant statistical content. While emphasizing the relevance of using rigorous statistical and probabilistic techniques in software engineering contexts, it presents opportunities for further research in the statistical sciences and their applications to software engineering.

It is intended to motivate and attract new researchers from statistics and the mathematical sciences to attack relevant and pressing problems in the software engineering setting. It describes the "big picture," as this approach provides the context in which statistical methods must be developed. The book's survey nature is directed at the mathematical sciences audience, but software engineers should also find the statistical emphasis refreshing and stimulating. It is hoped that the book will have the effect of seeding the field of statistical software engineering by its indication of opportunities where statistical thinking can help to increase understanding, productivity, and quality of software and software production.

  1. ×

    Welcome to OpenBook!

    You're looking at OpenBook, NAP.edu's online reading room since 1999. Based on feedback from you, our users, we've made some improvements that make it easier than ever to read thousands of publications on our website.

    Do you want to take a quick tour of the OpenBook's features?

    No Thanks Take a Tour »
  2. ×

    Show this book's table of contents, where you can jump to any chapter by name.

    « Back Next »
  3. ×

    ...or use these buttons to go back to the previous chapter or skip to the next one.

    « Back Next »
  4. ×

    Jump up to the previous page or down to the next one. Also, you can type in a page number and press Enter to go directly to that page in the book.

    « Back Next »
  5. ×

    Switch between the Original Pages, where you can read the report as it appeared in print, and Text Pages for the web version, where you can highlight and search the text.

    « Back Next »
  6. ×

    To search the entire text of this book, type in your search term here and press Enter.

    « Back Next »
  7. ×

    Share a link to this book page on your preferred social network or via email.

    « Back Next »
  8. ×

    View our suggested citation for this chapter.

    « Back Next »
  9. ×

    Ready to take your reading offline? Click here to buy this book in print or download it as a free PDF, if available.

    « Back Next »
Stay Connected!