we chose data reflecting the “control,” that is, the most widely accepted scoring paradigm.
Indistinguishable datasets. In a very few (< 5) instances, multiple (usually two) datasets remained with none taking precedence on the above grounds. In these instances, the dataset most favorable to polygraph testing was used.
This stage of review was accomplished by a small subgroup of committee members, staff, and the consultant, under oversight of a committee member specializing in research methodology.
We have conducted a systematic review but not a meta-analysis. A meta-analysis is a systematic review that integrates the compiled results of either the totality of selected studies or homogeneous subgroups of them into one or a few simple numerical summaries, each of which usually addresses both statistical significance (e.g., p-value) and the magnitude of an observed relationship (effect size). The best meta-analyses also include a search for systematic explanations of heterogeneity in the results of the studies compiled. Initially proposed to overcome the sample size limitations of individual studies and misinterpretations of negative statistical hypothesis tests, meta-analysis has seen widespread application as a general tool for research synthesis in the social and health sciences. Others have made efforts to do meta-analyses for all or part of the literature on the use of the polygraph for the detection of deception or the presence of concealed information (e.g., see Kircher et al., 1988; Urban, 1999; and Ben-Shakhar and Elaad, 2002). We have not attempted such numerical reduction here. In view of the widespread expectation that critical literature reviews lead to such comprehensive summaries, we offer some explanation for this decision.
There are both technical and substantive reasons for not using meta-analytic methods in this report. We do not use these methods in part because the literature does not allow us to deal adequately with the heterogeneity of the available studies. The laboratory studies employ instruments measuring different physiological parameters, multiple scales of measurement and systems of scoring, varying methods of interviewing, examiners of different levels of experience, and multiple study populations. The field studies present all these kinds of heterogeneity and more: they include variation within studies in the deceptions of concern, in examiners’ expectancies, and in multiple unrecorded aspects of the social interaction during the polygraph examination. Appropriate meta-analytic summaries would handle this diversity either by hypothesizing that