National Academies Press: OpenBook

Summary of a Workshop on Software Certification and Dependability (2004)

Chapter: 1 Overview of Workshop Discussions

« Previous: Front Matter
Suggested Citation:"1 Overview of Workshop Discussions." National Research Council. 2004. Summary of a Workshop on Software Certification and Dependability. Washington, DC: The National Academies Press. doi: 10.17226/11133.
×

1
Overview of Workshop Discussions

This report summarizes a workshop on software certification and dependability held April 19-20, 2004, in Washington, D.C., under the auspices of the Committee on Certifiably Dependable Software Systems. Several items should be kept in mind when reading this report:

  • The workshop focused on the subset of areas that the committee believed would best help frame the program of work for the remaining study period. There are areas of direct relevance for the study that are missing from the workshop agenda, either because of time constraints or because the panelists chose to address different areas. The committee plans to gather input on those areas in subsequent activities; feedback and additional input from readers of this report are welcome.

  • During the workshop, committee members deliberately refrained from questioning views expressed at the workshop—they preferred to use the time to gather input from workshop participants in an impartial manner. In addition, the committee chose not to extend the discussions in this first-phase report, instead reserving that task for the final report. Consequently, this report does not provide a free-standing overview of the current state of software development, of certification, or of anything other than the views expressed at this particular workshop.

  • The panel summaries have not been edited to make the terminology used by each panel consistent across the entire report. Meanings should be clear from the context. Deciding on appropriate and consistent terminology is a task for the committee as it prepares its final report.

Listed below are the main themes arising from each panel session. These themes are not conclusions or findings of the committee; they incorporate ideas extracted from each panel that seem to represent the major thrusts of each discussion. Each panel session discussion is elaborated in Chapter 2.

PANEL A THE STRENGTHS AND LIMITATIONS OF PROCESS

  • While following particular processes cannot alone guarantee certifiably dependable software, comprehensive engineering processes are nevertheless important to achieving this goal.

Suggested Citation:"1 Overview of Workshop Discussions." National Research Council. 2004. Summary of a Workshop on Software Certification and Dependability. Washington, DC: The National Academies Press. doi: 10.17226/11133.
×
  • In evaluating a system, appropriate metrics should be used; it may be necessary to measure secondary artifacts (e.g., process quality) as surrogates if what is actually of interest cannot be measured.

  • Developing ways to determine how best to allocate resources (e.g., understanding where errors are likely to cluster) can improve both dependability and cost-effectiveness.

PANEL B LOOKING FORWARD: NEW CHALLENGES, NEW OPPORTUNITIES

  • Over the past several decades society has become increasingly dependent on software. While desktop systems are not generally regarded as safety-critical, these days they are often task-critical for their users. This is true not only at the individual level but also at the organizational level.

  • One increasingly sophisticated set of tools that can help in the software development process with respect to dependability is the set of tools related to programming languages, such as type checkers, static analyzers, and model checkers.

  • Systems integration is a growing and challenging problem. Additional tools and strategies are needed to cope with large-scale systems integration issues.

PANEL C CERTIFICATION AND REGULATION: EXPERIENCE TO DATE

  • The process of certification may add value in a collateral fashion because attention must be paid to issues that might not receive it otherwise; given that software and its uses and contexts change over time, any value that certification has decays over time as well.

  • Market forces and the cost structure of the software industry may create incentives to release flawed software.

  • Validation—determining what the software should do—is often harder than verification, or determining whether the software does it correctly, and may be more important. Despite the difficulties of achieving validation systematically, however, many critical systems seem to function well.

PANEL D ORGANIZATIONAL CONTEXT, INCENTIVES, SAFETY CULTURE, AND MANAGEMENT

  • Systems are certified only within a particular context and up to specified system boundaries; certification of a system may not guarantee the dependability and usefulness of a system over its lifetime.

  • As a system’s reliability increases or is demonstrated over long periods of time, dependence on that system may increase to an extent not anticipated in the original design.

  • Accountability, reporting, and communication are difficult issues that must be planned and managed in detail across an organization.

Suggested Citation:"1 Overview of Workshop Discussions." National Research Council. 2004. Summary of a Workshop on Software Certification and Dependability. Washington, DC: The National Academies Press. doi: 10.17226/11133.
×

PANEL E COST-EFFECTIVENESS OF SOFTWARE ENGINEERING TECHNIQUES

  • There are interesting substantive overlaps in approaches to software development that seem philosophically opposed on the surface. In particular, agile methods such as “Extreme Programming” seem to share important elements with methods that employ formal notations for early modeling and analysis.

  • Understanding what is meant by “dependability” is critical; it was observed that, given its ubiquitous use and deployment, software is treated as though generally dependable.

  • Achieving dependable, certifiable software will require emphasis on process, people, and tools.

PANEL F CASE STUDY: ELECTRONIC VOTING

  • Structural flaws in the voting system go beyond the absence of voter-verifiable paper trails.

  • The lack of detailed risk analysis, coupled with a lack of openness in the voting system certification process, poses serious challenges to achieving a dependable voting infrastructure.

  • The current certification process does not seem to have resulted in secure or dependable electronic voting systems.

Suggested Citation:"1 Overview of Workshop Discussions." National Research Council. 2004. Summary of a Workshop on Software Certification and Dependability. Washington, DC: The National Academies Press. doi: 10.17226/11133.
×

This page intentionally left blank.

Suggested Citation:"1 Overview of Workshop Discussions." National Research Council. 2004. Summary of a Workshop on Software Certification and Dependability. Washington, DC: The National Academies Press. doi: 10.17226/11133.
×
Page 1
Suggested Citation:"1 Overview of Workshop Discussions." National Research Council. 2004. Summary of a Workshop on Software Certification and Dependability. Washington, DC: The National Academies Press. doi: 10.17226/11133.
×
Page 2
Suggested Citation:"1 Overview of Workshop Discussions." National Research Council. 2004. Summary of a Workshop on Software Certification and Dependability. Washington, DC: The National Academies Press. doi: 10.17226/11133.
×
Page 3
Suggested Citation:"1 Overview of Workshop Discussions." National Research Council. 2004. Summary of a Workshop on Software Certification and Dependability. Washington, DC: The National Academies Press. doi: 10.17226/11133.
×
Page 4
Next: 2 Summary of Panel Sessions and Presentations »
Summary of a Workshop on Software Certification and Dependability Get This Book
×
Buy Paperback | $29.00 Buy Ebook | $23.99
MyNAP members save 10% online.
Login or Register to save!
Download Free PDF

Certification of critical software systems (e.g., for safety and security) is important to help ensure their dependability. Today, certification relies as much on evaluation of the software development process as it does on the system’s properties. While the latter are preferable, the complexity of these systems usually makes them extremely difficult to evaluate. To explore these and related issues, the National Coordination Office for Information technology Research and Development asked the NRC to undertake a study to assess the current state of certification in dependable systems. The study is in two phases: the first to frame the problem and the second to assess it. This report presents a summary of a workshop held as part of the first phase. The report presents a summary of workshop participants’ presentations and subsequent discussion. It covers, among other things, the strengths and limitations of process; new challenges and opportunities; experience to date; organization context; and cost-effectiveness of software engineering techniques. A consensus report will be issued upon completion of the second phase.

  1. ×

    Welcome to OpenBook!

    You're looking at OpenBook, NAP.edu's online reading room since 1999. Based on feedback from you, our users, we've made some improvements that make it easier than ever to read thousands of publications on our website.

    Do you want to take a quick tour of the OpenBook's features?

    No Thanks Take a Tour »
  2. ×

    Show this book's table of contents, where you can jump to any chapter by name.

    « Back Next »
  3. ×

    ...or use these buttons to go back to the previous chapter or skip to the next one.

    « Back Next »
  4. ×

    Jump up to the previous page or down to the next one. Also, you can type in a page number and press Enter to go directly to that page in the book.

    « Back Next »
  5. ×

    Switch between the Original Pages, where you can read the report as it appeared in print, and Text Pages for the web version, where you can highlight and search the text.

    « Back Next »
  6. ×

    To search the entire text of this book, type in your search term here and press Enter.

    « Back Next »
  7. ×

    Share a link to this book page on your preferred social network or via email.

    « Back Next »
  8. ×

    View our suggested citation for this chapter.

    « Back Next »
  9. ×

    Ready to take your reading offline? Click here to buy this book in print or download it as a free PDF, if available.

    « Back Next »
Stay Connected!