The Future of Statistical Software Proceedings of a Forum (1991) / Chapter Skim
Currently Skimming:
Incorporating Statistical Expertise into Data Analysis Software
Incorporating Statistical Expertise into Data Analysis Software
Pages 51-62
The Chapter Skim interface presents what we've algorithmically identified as the most significant single chunk of text within every page in the chapter.
Select key terms on the right to highlight them within pages of the chapter.
From page 51... ...
Not surprisingly, such ideas were not enthusiastically embraced by the statistics community. Few of the criticisms were legitimate, as most were concerned with the impossibility of automating the "art" of data analysis.
|
From page 52... ...
By encoding into software the statistician's expertise in data analysis, and by directing statisticians' infatuation with resampling methodology, there is now a unique opportunity to study the data analysis process itself. This will allow the operating characteristics of several tests applied in sequence -- or even an entire analysis, as opposed to the properties of a single test or estimator -- to be understood.
|
From page 53... ...
It is the process of putting together many tiny pieces, the process called data analysis, and is not really understood. Encoding these pieces provides a platform from which to study this process that was invented to tell people what to do, and about which little is known.
|
From page 54... ...
When building expertise into software it must be remembered that good data analysis relies on pattern recognition, and consequently graphics should be heavily integrated into the process. Most of what is seen cannot be simply quantified with a single number.
|
From page 55... ...
In all of the systems described below, there are various levels of this automation-interaction trade-off. Efforts to Build Data Analysis Software REX Going back into recent history, there was a system that Bill Gale and I were involved with called REX (circa 1982)
|
From page 56... ...
It had been hoped that the language provided in REX would be a fertile playground for others to bootstrap their own interests, whether in time series, cluster analysis, or whatever. We knew of no other way to build strategies.
|
From page 57... ...
. Student ~ was fairly confident that the only way to end up with strategies and plans for understanding the data analysis process was by recording and analyzing the working of examples.
|
From page 58... ...
It actually ran on a different computer than that which was running the statistical software. With a network between the two, there were numerous logistical problems that got in the way and made things very tedious.
|
From page 59... ...
For TESS, it was necessary to develop a vocabulary of what seemed to be important in describing regression data, i.e., the qualitative features of the data. These were not merely slopes and intercepts, but rather qualitative features closely linked with how data are represented and summarized.
|
From page 60... ...
With some modern statistical-computing languages' one can do interesting things within a system and not have to go outside it. Mini-expert Functions A final type of data-analysis software that possesses statistical expertise is a mini-expert function.
|
From page 61... ...
Finally, let me mention some interesting work that may stimulate readers who feel that incorporating software into data analysis plans is important. John Adams, a recent graduate from the University of Minnesota, wrote a thesis directed at trying to understand the effects of all possible combinations of the things that are done in regression for a huge designed experiment tAdams, 19903.
|
From page 62... ...
Gale, W.A., 1986b, Student -- Phase 1, in Artificial Intelligence and Statistics, W.A. Gale, ea., Addison Wesley, Menlo Park, Calif.
|
Key Terms
This material may be derived from roughly machine-read images, and so is provided only to facilitate research.
More
information on Chapter Skim is available.