Below is the uncorrected machine-read text of this chapter, intended to provide our own search engines and external engines with highly rich, chapter-representative searchable text of each book. Because it is UNCORRECTED material, please consider the following text as a useful but insufficient proxy for the authoritative book pages.
POSTSCRIPT
i Postscript We have been asked to respond to a letter from the committee raising questions about the presence and consequences of methodological flaws in the ganzfeld studies discussed by Honorton (1985), Hyman (1985), Hyman and Honorton (1986), Rosenthal (1986), and by the present authors of this paper. Our response is in two parts. In part 1, we examine the likely effects of flaws on the meta-analytic results of the ganzfeld studies. In part 2, we examine the results of a series of new studies designed to address the flaws discussed by Hyman and Honorton in their individual and joint papers. Flaw Effects The committee has called our attention to possible flaws in the randomization procedure employed by Sargent and his colleagues. In its letter it noted that Honorton agreed with Hyman about the assignment of these randomization flaws to the Sargent study. However, Honorton states in two letters that this agreement was not reached (personal communications of November 25, 1987, and January 15, 1988). Apparently, experts on the ganzfeld research disagree on whether the Sargent studies' randomization procedures are flawed given all the evidence available to both, evidence which is summarized in papers by Blackmore (1987), Harley and Matthews (1987), and Sargent (1987). For purposes of this postscript and the following data analyses, we are going to assume that Hyman is correct in his assignment of randomization flaws and all other flaws he assigned in his 1985 paper. The heart of the matter is the relationship of flaws to ..t' research results and that is what our analyses are designed to investigate. In a 1986 manuscript, Hyman suggested that the relationship of flaws to study outcomes should
2 be examined in a multivari ate manner . According ly, that is the nature of our analyses in our first pass effort to examine the likelihood that methodological flaws are driving the results of the ganzfeld studies to an appre ciable degree . Canonical analysis. The most general of the multi~rariate procedures examines the maximum relationship that can be found between two sets of variables, for example, a set of predictor Variables and a set of outcome variables. In our analysis the predictor variables were Hyman's (1985) flaw variables of documentation (DOC), feedback (FB), randomization (R), security (SEC), single target (ST), and statistical analysis (STAT), all coded as O if adequate or 1 if not adequately done or not adequately specified. lye outcome variables were the significance level Z and the effect size Cohen's _. Me adjusted canonical correlation was only .46, a magnitude that for two-predicted-from-six could have arisen under the null hypothesis 54 times out of 100 (F(12,40) = 0.91). Interestingly, three of the six flaw variables correlated positively with the flaw canonical variable and with the outcome canonical variable (DOC, FB, R) but three correlated negatively (SEC, ST, STAT). thus, the canonical analysis gives no support to the hypothesis that the research results are a significant faction of the set of flaw variables. Regression analysis. Separate analyses were also done for each of the outcome variables Z and h. The battery of predictor variables correlated only .44 with Cohen's h (F(6,21) = 0.84, p = .56) and .57 with Z (F(6,21) = 1.65, p = .183. For neither of ache outcome variables did any of the six predictors account for a significant proportion of variance either in zero-order form or after partialing. Since there were two methods of pa~ialing employed, a total of 36 (3 methods x 6 predictors
3 l x 2 outcome variables) it's were computed, none of which reached the .05 i, level. Regression analyses, therefore, gave no more support than did the canonical analysis to the hypothesis that ganzfeld research results are a significant function of the set of flaw variate les . New Evi dence Hyman (1985) and Honorton (1985) were agreed tHyman and Honorton, 1986) that new studies were needed that would take account of the flaws they had found in their critiques of earlier research. Since our present paper was completed we have learned of a series of 10 new studies conducted by Honorton, one of the four investigators singled out by the Committee on Techniques for the Enhancement of Human Performance as among the best in the country (Druckman and Swets, 1988, p. 223. Ibe series of 10 ganzfeld studies yielded a combined Z of 2. 79, ~ = .0026 and amean h of .23. This effect size, based on 10 studies, is only slightly smaller than the mean effect size of Sargent's nine studies (h = .30) and is very close to the mean effect size of the remaining 19 studies (h = . 26; see Table 4a). For the original 28 studies plus the 10 new ones from Honorton's lab, the combined _ is now 7.10 and the mean effect size is now an h of .27. knitting Sargent's nine studies changes matters very little-- _ is now 5.74 and h = .25. In short, the new evidence based on studies designed to meet earlier methodological objections is very consistent with the earlier evidence and makes the null hypothesis still more implausible. Conclus ion Our analysis of the effects of flaws on study outcome lends no support to the hypothesis that ganzfeld research results are a significant function of the set of flaw variables. In addition, a series of 10 new studies designed to control for earlier presumed flaws yielded results quite consistent with the original set of 28 studies.
4 References Blacicmore, S. (1987) . A report of a visit to Carl Sargent's laboratory. Journal of the Society for Psychical Research, 54, 186-198. . . Druckman, D., ~ Swets, J. A. (Eds . ) (1988) . Issues, theories, and techniques. Press . Harley, T., ~ Matthews, G. (1987). science: A reply to Blackmore. Research, _, 199 -207 . Enhancing human performance. Washington, D . C .: National Academy Cheating, psi, and the appliance of Journal of the Society for Psychical Hyman, R. (19863. To conclude or not to conclude: A reply to the commentators. Unpublished manuscript, University of Oregon. Hyman, R., ~ Honorton, C. (19863. A j Dint communique: controversy. Jou~-~al of Parapsychology, 50, 351-364. - Rosenthal, R. ( 1986) . replication: 315 -336 . the psi ganzfeld Meta-analytic procedures and the nature of The ganzfeld debate. Journal of Parapsychology, 50, . Sargent, C. (1987). Sceptical fairytales from Bristol. Journal of the Society for Psychi Cal Re search, 54, 208-218 . . .