used to create a file used as input to an analysis program may fall under this definition. Exactness consists of a choice of an acceptable algorithm and the accurate implementation of the algorithm. For example, asymptotic formulae for variance estimation or p-value calculations should not be used in small samples whenever more precise calculations are available. The algorithm should either tolerate stressful data configurations, or detect them and gracefully report an inability to proceed. Guidance consists of the help provided by the structure and features of the software in conducting a correct and effective analysis. This includes assistance in choosing appropriate instructions, as well as assistance in deciding whether a particular approach is valid. Richness consists of how fully the software can do the analysis. The term coverage may be preferred by some. Throughout, the user will be taken to refer to the person executing the software.

The definitions of exactness, guidance, and richness all overlap somewhat. Distinctions between guidance and richness are particularly important in defining the boundaries of the present discussion. At one extreme, providing maximal guidance leads to an expert-system approach. In that case, richness ceases to exist as a separate property, being determined by the range of tolerated inputs and guidance accuracy. At the other extreme, providing minimal guidance leads to documenting features of the software and nothing else. An intermediate amount of guidance would be the automatic inclusion of diagnostics concerning the assumptions of the method of analysis. In contrast, richness describes the availability and convenience of creating such diagnostics. In considering guidance, the validity of the analysis approach for the data at hand is always in question. In contrast, when discussing richness it will be assumed throughout that using the software for the data at hand is a valid endeavor.

The one-way layout in analysis of variance (ANOVA) is used as the basic example. For the sake of brevity, familiarity with traditional approaches will be assumed. Kirk [1982] provided a comprehensive treatment of a large range of ANOVA models. In one-way ANOVA, a continuous response variable is examined to assess whether it is related to a categorical predictor variable. The predictor values may be strictly nominal, ordered categories, or interval scale values. Typically two to ten distinct predictor values are present. A number of regularity conditions must be assumed for both non-parametric as well as parametric methods in order to ensure the validity of statistical analysis. These may be loosely grouped into assumptions concerning (1) existence, (2) independence, (3) model, and (4) distribution. Traditional parametric fixed-effect ANOVA requires independent and identically distributed (i.i.d.) Gaussian scores within each category and categories that differ only by expected value.

Richness Dimensions

1. Epistemological Goals

Definition The epistemological goal(s) for an analysis consists of the standards by which



The National Academies | 500 Fifth St. N.W. | Washington, D.C. 20001
Copyright © National Academy of Sciences. All rights reserved.
Terms of Use and Privacy Statement