**Suggested Citation:**"References." National Research Council. 1996.

*Statistical Software Engineering*. Washington, DC: The National Academies Press. doi: 10.17226/5018.

**References**

Abdel-Ghaly, A.A., P.Y. Chan, and B. Littlewood. 1986. Evaluation of competing software reliability predictions. *IEEE Trans. Software Eng.* **SE-12**(9):950-967.

Abdel-Hamid, T. 1991. *Software Project Dynamics: An Integrated Approach* . Englewood Cliffs, N.J.: Prentice-Hall.

*American Heritage Dictionary of the English Language, The**.* 1981. Boston: Houghton Mifflin.

American Statistical Association (ASA). 1993. *Combining Information: Statistical Issues and Opportunities for Research*, Contemporary Statistics Series, No. 1. Alexandria, Va.: American Statistical Association.

Baecker, R.M. and A. Marcus. 1988. *Human Factors and Typography for More Readable Programs*. Reading, Mass.: Addison Wesley.

Baker, M.J. and S.G. Eick. 1995. Space-filling software displays. *J. Visual Languages Comput.* **6**(2). In press.

Basili, V. 1993. Measurement, analysis and modeling, and experimentation in software engineering. Unpublished paper presented at Forum on Statistical Methods in Software Engineering, October 11-12, 1993, National Research Council, Washington, D.C.

Basili, V. and D. Weiss. 1984. A methodology for collecting valid software engineering data. *IEEE Trans. Software Eng.* **SE-10**:6.

Becker, R.A. and W.S. Cleveland. 1987. Brushing scatterplots. *Technometrics* **29**:127-142.

Becker, R.A., W.S. Cleveland, and A.R. Wilks. 1987. Dynamic graphics for data analysis. *Statistical Science* **2**:355-383.

Beckman, R.J. and M.D. McKay. 1987. Monte Carlo estimation under different distributions using the same simulation. *Technometrics* **29**:153-160.

Blum, M., M. Luby, and R. Rubinfeld. 1989. Program result checking against adaptive programs and in cryptographic settings . Pp. 107-118 in *Distributed Computing and Cryptography*, J. Feigenbaum and M. Merritt, eds. DIMACS: Series in Discrete Mathematics and Theoretical Computer Science, Vol. 2. Providence, R.I.: American Mathematical Society.

Blum, M., M. Luby, and R. Rubinfeld. 1990. Self-testing/correcting with applications to numerical problems. *STOC* **22**:73-83.

Boehm, B.W. 1981. *Software Engineering Economics*. Engelwood Cliffs, N.J.: Prentice Hall.

Brocklehurst, S. and B. Littlewood. 1992. New ways to get accurate reliability measures. *IEEE Software* **9**(4):34-42.

Brown, M.H. and J. Hershberger. 1992. Color and sound in algorithm animation. *IEEE Computer* **25**(12):52-63.

Burnham, K.P. and W.S. Overton. 1978. Estimation of the size of a closed population when capture probabilities vary among animals. *Biometrika* **45**:343-359.

Chillarege, R., I. Bhandari, J. Chaar, M. Halliday, D. Moebus, B. Ray, and M. Wong. 1992. Orthogonal defect classification-A concept for in-process measurements. *IEEE Trans. Software. Eng.* **SE***-***18**:943-955.

Cohen, D.M., S.R. Dalal, A. Kaija, and G. Patton. 1994. The automatic efficient test generator (AETG) system. Pp. 303-309 in *Proceedings of the 5th International Symposium on Software Reliability Engineering*. Los Alamitos, Calif.: IEEE Computer Society Press.

**Suggested Citation:**"References." National Research Council. 1996.

*Statistical Software Engineering*. Washington, DC: The National Academies Press. doi: 10.17226/5018.

Curtis, W. 1988. The impact of individual differences in programmers. Pp. 279-294 in *Working with Computers: Theory versus Outcome**,* G.C. van der Veer et al., eds. San Diego, Calif.: Academic Press.

Dalal, S.R. and C.L. Mallows. 1988. When should one stop software testing*?* *J. Am. Statist. Assoc*. **83**:872-879.

Dalal, S.R. and C.L. Mallows. 1990. Some graphical aids for deciding when to stop testing software. *IEEE J. Selected Areas in Communications* **8**:169-175. (Special issue on software quality and productivity.)

Dalal, S.R. and C.L. Mallows. 1992. Buying with exact confidence. *Ann. Appl. Probab*. **2**:752-765.

Dalal, S.R. and A.M. McIntosh. 1994. When to stop testing for large software systems with changing code. *IEEE Trans. Software Eng.* **SE****20**:318-323.

Dalal, S.R., E.B. Fowlkes, and A.B. Hoadley. 1989. Risk analysis of the space shuttle: Pre-Challenger prediction of failure. *J. Am. Stat. Assoc*. **84**:945-957.

Dalal, S.R., J.R. Horgan, and J.R. Kettenring. 1994. Reliable software and communication II: Controlling the software development process. *IEEE J. Selected Areas in Communications* **12**:33-39.

Dalkey, N.C. 1972. *Studies in the Quality of Life-Delphi and Decision-Making**.* Lexington, Mass.: D.C. Heath & Co.

Dawid, A.P. 1984. Statistical theory: The prequential approach. *J. R. Stat. Soc. London* *A* **147**:278-292.

DeMillo, R.A., D.S. Guindi, K.S. King, W.M. McCracken, and A.J. Offutt. 1988. An extended overview of the MOTHRA mutation system. Pp. 142-151 in *Proceedings of the Second Workshop on Software Testing, Verification and Analysis*. Alberta, Canada: Banff.

Ebert, C. 1992. Visualization techniques for analyzing and evaluating software measures. *IEEE Trans. Software Eng.* **11**(18):1029-1034.

Eckhardt, D.E. and L.D. Lee. 1985. A theoretical basis of multiversion software subject to coincident errors. *IEEE Trans. Software Eng.* **SE-11**:1511-1517.

Eckhardt, D.E., A.K. Caglayan, J.C. Knight, L.D. Lee, D.F. McAllister, M.A. Vouk, and J.P. Kelly. 1991. An experimental evaluation of software redundancy as a strategy for improving reliability. *IEEE Trans. Software Eng.* **SE-17**(7):692-702.

Eick, S.G. 1994. Graphically displaying text. *J. Comput. Graphical Stat.* **3**(2):127-142.

Eick, S.G., C.R. Loader, M.D. Long, S.A. Vander Wiel, and L.G. Votta. 1992a. Estimating software fault content before coding. Pp. 59-65 in *Proceedings of the 14th International Conference on Software Engineering* (Melbourne, Australia). Los Alamitos, Calif.: IEEE Computer Society Press.

Eick, S.G., J.L. Steffen, and E.E. Sumner. 1992b. A tool for visualizing line oriented software. *IEEE Trans. Software Eng.* **11**(18):957-968.

Ganser, E.R., E.E. Koutsofios, S.C. North, and K.-P. Vo. 1993. A technique for drawing directed graphs. *IEEE Trans. Software Eng.* **SE**-**19**(3):214-230.

Halstead, M.H. 1977. *Elements of Software Science**.* New York: Elsevier.

Hastie, T.J. and R.J. Tibshirani. 1990. *Generalized Additive Models* . London: Chapman & Hall.

**Suggested Citation:**"References." National Research Council. 1996.

*Statistical Software Engineering*. Washington, DC: The National Academies Press. doi: 10.17226/5018.

Henrion, M. and B. Fischhoff. 1986. Assessing uncertainty in physical constants. *Am. J. Phys.* **54**(9):791-798.

Horgan, J.R. and S. London. 1992. ATAC: A data flow testing tool for C. Pp. 2-10 in *Proceedings of the Second Symposium on Assessment of Quality Software Development Tools* (May 27-29, 1992, New Orleans, La.), E. Nahouraii, ed. Los Alamitos, Calif.: IEEE Computer Society Press.

Humphrey, W.S. 1988. Characterizing the software process: A maturity framework. *IEEE Software* **5**:73-79.

Humphrey, W.S. 1989. *Managing the Software Process**.* Reading, Mass.: Addison Wesley.

Iman, R.L. and W.J. Conover. 1982. A distribution free approach to inducing rank correlations among input variables. *Commun. Stat., Part B* **11**:311-334.

Institute of Electrical and Electronics Engineers (IEEE). 1990. *IEEE Standard Glossary of Software Engineering Terminology**.* IEEE Std. 610.12-1990. New York: IEEE, Inc.

Institute of Electrical and Electronics Engineers (IEEE). 1993. *IEEE Standard for Software Productivity Metrics*. IEEE Computer Society, IEEE Std. 1045-1992, January 11, 1993. New York: IEEE, Inc.

Ishikawa, K. 1976. *Guide to Quality Control*. Tokyo, Japan: Asian Productivity Organization.

Kahneman, D., P. Slovic, and A. Tversky, eds. 1982*.* *Judgment Under Uncertainty: Heuristics and Biases*. New York: Cambridge University Press.

Keller, T.W. 1993. Maintenance process metrics for space shuttle flight software . Unpublished paper presented at Forum on Statistical Methods in Software Engineering, October 11-12, 1993, National Research Council, Washington, D.C.

Kitchenham, B. 1991. Never mind the metrics; what about the numbers! Pp. 28-37 in *Formal Aspects of Measurement**,* T. Denvir, R. Herman, and R.W. Whitty, eds. Proceedings of the BCS-FACS Workshop, May 5, 1991, South Bank University, London. New York: Springer-Verlag.

Kitchenham, B. 1992. *Analyzing Software Data*. Metrics Club Report. Manchester, England: National Computing Centre, Ltd.

Knight, J.C. and N.G. Leveson. 1986. Experimental evaluation of the assumption of independence in multiversion software. *IEEE Trans. Software Eng.* **SE-12**(1):96-109.

Lee, D. and M. Yanakakis. 1992. On-line minimization of transition systems. Pp. 264-274 in *Proceedings of the 24th Annual ACM Symposium on Theory of Computing*. New York: Association for Computing Machinery.

Leveson, N.G. 1986. Software safety: why, what, and how. *ACM Comput. Surveys* **8**:125-163.

Lipton, R. 1989. New directions in testing. Pp. 191-202 in *Distributed Computing and Cryptography,* J. Feigenbaum and M. Merritt, eds. DIMACS: Series in Discrete Mathematics and Theoretical Computer Science, Vol. 2. Providence, R.I.: American Mathematical Society.

Littlewood, B. 1979. Software reliability model for modular program structure. *IEEE Trans. Reliability* **R-28**(3):241-246.

Littlewood, B. and D.R. Miller. 1989. Conceptual modeling of coincident failures in multi-version software. *IEEE Trans. Software Eng.* **SE-15** (12):1596-1614.

**Suggested Citation:**"References." National Research Council. 1996.

*Statistical Software Engineering*. Washington, DC: The National Academies Press. doi: 10.17226/5018.

Littlewood, B. and L. Strigini. 1993. Validation of ultra-high dependability for software-based systems. *Communications of the Association for Computing Machinery* **36**(11):69-80.

Mallows, C.L. 1973. Some comments on Cp. *Technometrics* **15**:661-667.

McCabe, T.J. 1976. A complexity measure. *IEEE Trans. Software Eng.* **SE-1**(3):312-327.

McKay, M.D., W.J. Conover, and R.J. Beckman. 1979. A comparison of three methods for selecting values of input variables in the analysis of output from a computer code. *Technometrics* **21**:239-245.

Mosteller, F. and J.W. Tukey. 1977. *Data Analysis and Regression: A Second Course in Statistics*. Reading, Mass.: Addison Wesley.

Munson, J.C. 1993. The relationship between software metrics and quality metrics. Unpublished paper presented at Forum on Statistical Methods in Software Engineering, October 11-12, 1993, National Research Council, Washington, D.C.

National Research Council (NRC). 1991. *The Future of Statistical Software*. Committee on Applied and Theoretical Statistics, Board on Mathematical Sciences. Washington, D.C.: National Academy Press.

National Research Council (NRC). 1992. *Combining Information: Statistical Issues and Opportunities for Research*. Committee on Applied and Theoretical Statistics, Board on Mathematical Sciences. Washington, D.C.: National Academy Press. (Reprinted in 1993 by the American Statistical Association as Volume 1 in the ASA Contemporary Statistics series.)

Nayak, T.K. 1988. Estimating population size by recapture sampling. *Biometrika* **75**:113-120.

Phadke, M.S. 1993. Robust design method for software engineering. Unpublished paper presented at Forum on Statistical Methods in Software Engineering, October 11-12, 1993, National Research Council, Washington, D.C.

Pill, J. 1971. The Delphi method: Substance, context, a critique and an annotated bibliography . *Socio-Economic Planning Science* **5**:57-71.

Randell, B. and P. Naur, eds. 1968. *Software Engineering Concepts and Techniques**.* NATO Science Committee, Proceedings of the NATO Conferences, October 7-11, 1968, Garmisch, Germany. New York: Petrocelli/Charter.

Sackman, H. 1970*.* *Man-Computer Problem-Solving: Experimental Evaluation of Time-Sharing and Batch Processing**.* New York: Auerbach.

Siegrist, K. 1988a. Reliability of systems with Markov transfers of control. *IEEE Trans. Software Eng.* **SE-14**(7):1049-1053.

Siegrist, K. 1988b. Reliability of systems with Markov transfers of control, II*.* *IEEE Trans. Software Eng.* **SE-14**(10):1478-1480.

Singpurwalla, N.D. 1991. Determining an optimal time interval for testing and debugging software*.* *IEEE Trans. Software Eng.* **17**(4):313-319.

Smith, A.F.M. and G.O. Roberts. 1993. Bayesian computation via the Gibbs sampler and related Markov chain Monte Carlo methods. *J. R. Stat. Soc. London* *B* **55**(1):3-23.

Stasko, J. 1993. Software visualization. Unpublished paper presented at Forum on Statistical Methods in Software Engineering, October 11-12, 1993 , National Research Council, Washington, D.C.

**Suggested Citation:**"References." National Research Council. 1996.

*Statistical Software Engineering*. Washington, DC: The National Academies Press. doi: 10.17226/5018.

Stein, M. 1987. Large sample properties of simulations using Latin hypercube sampling. *Technometrics* **29**:143-151.

Tukey, J.W. 1977. *Exploratory Data Analysis*. Reading, Mass.: Addison Wesley.

Tukey, J.W. 1991. Use of many covariates in clinical trials. *Int. Stat. Rev.* **59**(2):123-128.

Vander Wiel, S.A. and L.G. Votta. 1993. Assessing software designs using capture-recapture methods. *IEEE Trans. Software Eng.* **SE***-***19**(11):1045-1054.

Zuse, H. 1991. *Software Complexity: Measures and Methods**.* Berlin: de Gruyter.

Zweben, S. 1993. Statistical methods in a study of software re-use principles. Unpublished paper presented at Forum on Statistical Methods in Software Engineering, October 11-12, 1993, National Research Council, Washington, D.C.