Below is the uncorrected machine-read text of this chapter, intended to provide our own search engines and external engines with highly rich, chapter-representative searchable text of each book. Because it is UNCORRECTED material, please consider the following text as a useful but insufficient proxy for the authoritative book pages.
Cntique of Survey Methodology INTRODUCTION When society is faced with a proposition that something, usually something new, is potentially harmful, it can take either (or both) of two opposing positions. One position, the null hypothesis, is that the effect (harm) cannot be assumed until there is scientific evidence of it. The other position is that potential harm should be assumed until there is proof that it does not exist. While at first glance the latter position may appear attractive, it presents problems. Demonstrating with absolute certainty that something produces no harm is an impossible task both logically and practically. How many years and dollars should be spent to try to rule out the infinite number of all possibilities for how VDTs- - r any technologymight be harmful in order to prove no harm? There are literally hundreds of mental and physical disease cate- gories that would need to be examined specifically by careful scientific analysis. And there would be an almost endless number of properties of VDTs (or other devices) that might have to be studied. A more practical approach calls on society to suggest accep- table limits regarding certain properties of VDTs and possible health risks. These limits should be specified on the basis of scientifically grounded theory rather than on the basis of intui- tion. Without such an approach, society would be placed in a position of responding to the advocacy of any group of people who want to ban or regulate something, whether or not there is any valid evidence to justify their concerns. Thus. Preliminary , . , findings suggesting harm or assertions of harm should be carefully examined to determine what competing explanations might be possible. For VDTs, as for any thing to be studied, research should be designed to develop and test competing hypotheses about effects: Is there an effect? Is it harmful? What are the causes and mechanisms? 30
31 Tests of these hypotheses can be made in many ways. One option is to conduct carefully controlled experiments; some of the research surveyed by the panel has indeed been experimental, and it is reviewed in other sections. Another option is to conduct field surveys; most of the research seems to have followed this option. Experiments offer undeniable advantages. Using well-designed experiments, one can control competing explanatory variables by randomly assigning people to conditions that vary only on the variable hypothesized to be causal: for example, one could randomly assign people to either data entry or data acquisition work and, within those conditions, to a group either using or not using VDTs. Most carefully controlled experimental research also has-some disadvantages, however. Compared with survey research, the cost of data collection per respondent is high. Special laboratory con- ditions must be created just to collect the data, and only a limited number of subjects can occupy such facilities at any time. Conse- quently, large sample data bases cannot be economically gener- ated in terms of time and financial costs. Another disadvantage is that most carefully controlled research, by the act of establishing the controls, creates an artificial situation that may not generalize to typical working environments. Using college sophomores in a VDT experiment may be convenient, but such research subjects do not worry about the loss of job security through automation, nor do they experi- ence the excitement of meeting a new challenge on the job. They do not find themselves in a changed career situation to which they may be resistant, nor do they have the choices or variety of tasks that might characterize a real job. Consequently, the results may not generalize to people who choose jobs with VDTs over jobs without such technology or to people who are on jobs they have already learned to perform without VDTs--in short, the results of such experiments may not generalize to real people in real jobs. The survey approach avoids this problem, but it has the disadvantage of being unable fully to control competing causes of effects by randomization. (For general examinations of survey methods, see Rosenberg, 1968; Warwick and Lininger, 1975.) The main advantage of field research is the realism of the phenomena it studies. _ a SURVEYS OF VDT USERS The aim of field research on VDT use is to describe unique and nonunique concomitants of the use of video display terminals. The question is whether one can identify visual, perceptual, or other
32 health effects of work in which VDTs are used. When such effects can be described, one needs to determine how much of the effect is uniquely proportional to the level of VDT use and how much is due, alternatively, to some other job, personal, or organizational characteristic that accompanies the use of VDTs. Some health outcomes do show up in the VDT literature. For example, VDT use is sometimes related to reports of stress or health complaints (including visual or postural problems). The complaints appear more often at the lower job levels; workers at such jobs are also more likely than other workers to report social complaints not directly related to the VDT itself--for example, low staff support, low cohesiveness, or ambiguity in relations with supervisors. Some studies find that visual complaints are more likely to be reported by workers whose work is limited to data entry than by those who may have VDT activity interrupted by work with customers or who work in an interactive mode. When negative health effects of this sort do appear, they usually sug- gest an alternative hypothesis: that they may be caused by correlated characteristics of the work situation rather than by characteristics of the VDT itself. Table 2.1 summarizes six studies comparing VDT work to non-VDT work or examining a range of VDT exposure. A critical appraisal of these studies indicates that they are uneven with regard to a number of basic criteria for drawing strong inferences from data (whether experimental or survey). Table 2.1 does not include studies that examined users of VDTs with no comparison group or with no variation in exposure to VDTs. In studies that lack a comparison group (or some variation between persons in exposure to VDT properties of interest), one has no idea if the level of complaints would be higher or lower than it would be in a group not exposed to VDTs. For example, some studies only ask users of VDTs what bothers them. While this might make interesting casual reading, it is of little scientific value because one does not know if the results would be the same or different for a group of respondents who did not use VDTs. An overview of studies involving comparison groups as well as studies using other approache`-one hesitates to call them design~can be found in Dainoff (1982~. Adequacy of Theory The lack of a theory specifying major constructs and the links between them is one of the most critical deficits in the survey research on VDTs. Without such theory, investigation becomes shotgun empiricism, and the risk of wasting effort trying to explain chance findings increases. Without such theory, the
33 choice of stressors to study, the possible mechanisms by which they produce strain, and the possible strain produced becomes an act of intuition, for which science claims no unique talent. Adequacy of Research Design Almost all the studies are cros~sectional surveys. Such designs do not allow one to determine what is antecedent and what is consequent in studies of VDT use and well-being. Investments in longitudinal panel designs will be required to make full use of some of the causal structural modeling techniques that have been developed for multivariate nonexperimental studies (see, e.g., Joreskog and Sorbom, 1979~. A good survey design for studying VDTs and well-being should make use of psychological, demographic, and situational controls. Then, if one finds differences in well-being as a function of VDT use, one can take steps to rule out characteristics of the operator, the content of the job, and the social and physical nature of the work setting in case they are confounded in part with VDT use. To consider job content and job setting, one needs an adequate job analysis. To consider individual differences, one needs an adequate assessment of such variables as employee motivation and skills. Such data have rarely been collected in VDT studies to date, especially survey studies of the well-being of VDT workers. There is a tendency for many survey studies to pigeonhole \/DT users and treat VDT work as a dichotomy: either one uses a VDT or one does not. More properly, some investigators view VDT work along a continuum and preserve valuable information about variations among individuals in exposure to VDT work. The tendency to pigeonhole can also occur in attempting to classify VDT use as either data entry or data acquisition or interactive and so on. Valuable data may be thrown out need- lessly by this procedure. For example, in interactive computer work, there may be some value in studying the percentage of each employee's work that is input, retrieval, creative, or noncreative. Adequacy of Measurement Some VDT research uses self-report measures of unknown or indeterminate reliability or validity. Studies should use multiple indicators of a condition to increase internal validity. (See Nunnally t19673 for a discussion of measurement theory and the importance of multiple indicators.) Some VDT research uses vague, open-ended questions about work as the primary source of data. Although such a procedure is
34 .= o o Cal ~ ~ _ o ~ (Q ~ ~ C, ._ s Is: Ce U. D4 ~ o a o ~ O ,- ~ a.) Can ~ ~ o o U) o Cal in A) > CO r' - ~5 0) En o £ £ En - C:L Cal to ._ - C~ of: ,_ U) - - ;e U. a, ~ ._ · US _ ~ ~ o ° At U. ~ o 3 ~ ~ a, ~ En - i .~-U' ~ ~ I :^ U. - C~ A: o ·3 Cal a) U. U. ~0 _ ~ ~ O _ ~ . ~ - - u) o c~ a., ~ :e o ~ ~ s .m 4, ~ 3 o _ ·;> .D =^ ~ ~ ~ ~ ~ ° C ._, s ~ .Y ~ Cq U. ° 'n 0 ~ a~ ~ ~ s C°~ ~ ;> s C: _ _ ~ o o _ ._ CO ~ ~ s ~ 3 _ ~ o U. o _ ._ ~ a o 3 0~3 C~ ~ ~ O O S 4,) .U~ (o ·U ~ U ~ ~ C D c~ c~ u, o _ ¢D _ ~ ;> ~ ~ ~ .= a~ ~ ~ S C: S O O ~ Ce S ~ {_ CtS °O ;5 · (D O O _ O U. ~ ~ ~ ~ ~. = . ~ ~ a) ~ ~ ~ u' _ S S ~ ~ ~ 3 0 _ O :~-_ Ce ~ CK5 ,C) ~ ~ ~ ~ ,= ~ _' ~ ~ ~ R: 3 o CO - C) c~ ~ a O ~ ~ ~ O ~ O Q0 ~ ~) oo 3 c) O ._ O .0 ~ O D C: ~L V~ ~ ~- =) 3 ~ O O o ~ 0, ~ ~ 3 3 _ - ~ c,, ~ ,= ~ Ct5 ~ ~ ~ ~ ~ ~ ~ O C~ V, :^ U, - o ._ - C~ U' U, U) o V - C~ _ ^ _ oo S O~ .~ _ ~ `:; V,
35 o ~ .O As, c) · E- _ ~ o ~ . co cts o o _. ._ _ ~ ·- U' Cat o o ~ s I. - a) s ~ ~ ,= o an ct AL ~ o do ~ = - in o o ~ ~ ·C~ Cs C) =^ ° ~ ~ ° EM .= on 3 ._ O4 ~ ~ c, o C ~ ~ ace ·, ~ CC o o ~ ~ ~ ~ 3° ° CO a.) a, ~ ~ U. ~ ·O ce 3 ~ c: ~ Its ° ~ ~ .o ~ ,, o ~ - - ~ O O D c, Ct ~ ~ ·~ U~ t~s ~ ~ O O -, _ =, u' o D 3 C) O ~ ~ ~ ~ O g ~ =~ - ;> ~ ~ · O O _ c, ~ O '~ ~ S ~ .~ .= ~ t~ ~ ~ t>O O e5 ,,~ .~ ~ C~ E 0 3 0 E e 5 c ~ ~ c) u' ~ ~ :^ ~ E C1 ~ ° 0 ~ .Y~ c `, ~ 0 oo 3 0 - .= ~ V ~ ~ 0 Z ~ 0 . . o ~ ~ ~ 11 ~ _ ·_ ^~ _O ~ _ ,,= >, ~ O_ C: ct c 3 ~ ° ° E ae r c~ v, :^ ;> u, ct 0 ._ . ~ cq u, u, ~o V ~: 0 0 ~ - - '- ~ v, o u, ao :5 . - s - ^ v o c: oo ~ c~ ~ ~ s ~ ~ c~ 3 3 ~ _ ,,L,~ s v a' ~ ,~^ ~ 3 ~ ~ c~ u, ~ ° ~ u, 3 - 0 u, u, ~ .O 0 a~ 0 _ ._) ._ . ~ u, ~D . c~ c~ ~ ~ s u, c°) ~ C;, 0 ~ u, ~ v _ _ ~ s~ ~ S~ ce _ ~ a' _ C) ~ ~ C) ° ~ ° s ~ .° c, e 8 ° ~ ° 3 c = ._ ~ v u, ~ ..O ~ x O G S ~D ~ S _ ~ E _ c, Z .° 5 Y . <, ~ 8, ~ E c 3-_ _: °, ~3 E3 c ~ ~ ~, ~ c O a :3 ·_ R
36 ~ C~ o o _ o V, V C) ._ s C~ U oo o o ~ ao o ~ v a~ cn o U. ~ Ct P~ C~ CQ .~ J oa ._ ~: o ._ - e~ ._ V o ~ o ;> ~ ._ ao c: a~ X (L) o · C: oo ·Y o 3 o" ~, 4,, s3 . _ s C<S ._ 3 o .= s .= Ct s ~ CC U~ s:: ._ O ~L) U. C~ :' "o oo ._ ._ oo U' :t 3 ~ 3 °-_ C ~.C O # C E 3 E ~ ~ ~ ~ ~ ~ ~ O ~ o ~ ~c R o = c~ a~ c~ :^ c~ - s - o u, a' - u, u' 'e a, . - - ~D o - ~: · e~ a~ . - s c) c~ c~s . - 3 U, 00 ~ 3 o y ~: U, - o - zo U, 8 =: o E~ ~ 3 ~ s c~ V ~ o '_ o ~ ~'~ ~ oo 3 e~ t: ._ ~ co U~ · >. (,, s ~ ~ _ . ~ ~ Q ~ 3 ~ a~ =` C ,= 3 3 C t~ C~ ·00 0 c~s ~ ~ _ S V, C~ ~ O .O 00 a P4 - e~ U, O ~ S O O S o _ ~0 S ~ O ~ ~ O S _ . C C C S~ c o. , a ~ ~ 3 a .= 0° (D S Ce C) 0 0 ,= S 0 oO Cd ~ ~ S r~ D ~ ~ O C c o a' ~ 3 ~> ~ 3 ~ 0~ ~ 3 ~ cd C o C ,C ~ tD C 0,) O ~ ¢~, L;~ ~ ~ O ~ C O. _~ ~ C;~ C) S ~ <~ ~ t~ ·~5, 5 · ;> O 2 ,,, C 0, ~ a c E c S O ~ t: O · ~ ~ ~ ~ U' ~ ~ S ~ =0 ^~ S ~ S V) y ~ ~ O O e ~ ~ E ~3 c, ~ 8 - 3 ~ <= '~ C ° =fi .E ~ d I ~ ' ' ~ d i · ' . ~ ~: . .
37 U. U3 o o .= U' C) ~ a.) ~ o ~ o . ~ ~ ~ oo ~ 3 a.) o Cal ~ As o Cal ~ ,` U. Cd ~ _ ~ ED ._ ._ 3 ~ 0 a, 'I i:: o ._ a, ~ cat Cal ~ ~ 3 ~ _ ~ ~ .'t 3 a' O ~ ~ U. To o ~ ~ .~= Al 3 "4 _ ~ ~ o ·= 3 '' o _t a., ~ ,_ C) .= o _ ~ ~ ~ 3 o .Cq Cs ~ so ~ . Z (L) ~ ~ ·" ~ =.= ~ 3 ~ EN . . Ct ;~ o U. _ _ it,, 3 3 cn 0 cn ~ 3 =^ 3 ~ o U. ~: ~ Ct oo ~ .= 9 (u s ~ U. oo ·~ ~ o _ C~ - ._ ._ o ._ ~_ C~ - C~ s ~o ~: C~ C~ U. - oo - - U, .= ~: ~s .O - o - V, C~ ~ . _ ~ C Ct 3 o ~: o - _l o ~, - o - zo ~, o . ~ ~ ~ a' O C.) Ce Ct ~ ~ ._ 'D o .= o _ ._ . _ ~ o _t Ct oo oo ~ ~ =, Ct ~ , Cq - C~ o - a' Cq U. o ~s ~_ _ o ~ oo C~ ~ .= o o C~ ._ Cd .O C~ · Ct - ~ e o 3 L. Co o . C~ ~ U~ o 1:5 oo aa = a, , ~ ~: o ~ - o ._ ~ ~ .° ." ~ ~ C) 3 "o o o Cd ~ _t _ C: Cs C~ ~ .~ . - o ~D _ o ·~ ~ :~.= o :^ 41) C~ o o ~ ~ · t* ~ ._ ._ ._ ~ ~ C) _ ~ ~ o ~ ~ L. ~ C~ ~ ._ Cq ~ ° ~ ~ o ~ C~ C~ ~ ~ ° CO) ·_ ~ ~ ~ ~ C ~ D z O C . ~: o ~: ~: o - - o z - o ~ - z ~ c ~ ~ ~ 3 ~ _ ~ 8 3 i~ ~ ° ~o = - ~ 1°1 o ;> V: Ct O _ _. - °e s ~ . _ C~ o o~.E ~ C: ~ ao ._ o ~ ~ o~ C) U, ao U, o
38 ~ u, 0 0 cat ~ _ 0 cry V c~ ._ ~s ct C,0 oo - o 0 ~ ~ ~o u, o ~ a., u, ~ c~ c~ v, ._ ._ cq a' c~ ~C C V cS ~ ~ ~ ~ ~ _ ~ ~ _ o D d) ~ ~ C C~ ,~3 C) = ~ ~ g ~ ~ ¢~ ~ ~ 3 ~ I ~ ~ §= ~Y ~ , ~ ¢- . e , ~ , ~ . E e o c ° ' s c 0 s 0 E c D s a c S C O ~ ~ S ~ o f,- DO E ~ ~s E 0 ° ° s ~ 5~ . , = ~ ~ . ~ ~ = 3 . a~ u~ 3 c: ._ _ o ~ c, v, a . ^^ E O 0 C; E ~ g _ c e ~ ~ s ~ ~ ~ `:: 2 s ~ s _ o ~ s ~ 0.°~> 9 3 3 o ~: ct, 0= ~ 0 ~ o ~ 0 _ oo a' ~ 8 v a' ~o oo .= 3 s:. ~o Ei ~ c~ c: v,
39 ~= ~ ~¢ 9 ~ a ' ~ ~ S ~ ~ , , ~ a ~ . . . . . . . . . . ~ ~ ~ ~ ~ ~ 00 cat 0 ~4 ~ _4 O c: a' TO -0 ._ _ ._ _ _ u, ~0 co O O u, as x sit O O 3 o 3 ~ _ sit 3 o O o Ct cam OF _ 'A E- _ ~ o ~ o U. _ ~ ,_CQ ._ l_ ~ o 4, 3 3 EN '= ~ ~ o _ _ v, ad .. ~ U' ~ o o x Z
40 useful in the early stages of research design to determine the language and range of responses people use in thinking about and describing their jobs, standardized scales should be used in the actual field survey. Standardized and focused questions and response scales are not as dependent on the verbal ability of respondents (the more loquacious respondents providing more content) as are open-ended questions. Standardized and focused questions tap areas of interest to the investigator rather than leaving it up to the respondent to decide what the investigator might think is important. While structured interviewing tech- niques with standardized scales may put words in the respondent's mouth, such an effect should be constant across different con- ditions of VDT use (unless one is prepared to argue otherwise on the basis of some compelling theory) and should, therefore, not affect the relative differences in intensity or frequency of response as a function of VDT exposure. Adequacy of Sampling Part of a good research design, of course, is the method by which one samples from populations of people and of environments. The samples in most of the studies in Table 2.1 appear to be ill-suited for drawing inferences to broad populations: 1. Diverse populations of employees are sometimes sampled and treated as if they are a similar pool of VDT users (e.g., professional and clerical VDT users in one group) without ascertaining if such pooling is empirically justified. 2. Diverse job environments are pooled (e.g., newsrooms and clerical offices). Measurement of illumination, glare, and other physical features of the workplace and VDT are not controlled for and are even unmentioned, despite their potential contributory role in a study of VDTs and well-being. 3. Comparisons of VDT groups and control or non-VDT groups may be confounded by differences between these groups in employee and job characteristics (see 1 and 2 above). Another serious problem in surveys of VDT users is the low response rate that occurs in many of the studies (and some studies do not even report response rates). Rates as low as 23 percent raise the possibility that the nonrespondents may be significantly different in key ways from the respondents. For example, if only those VDT users who are the most dissatisfied or experience the most symptoms are motivated to participate, and if this is not taken into account, one may seriously overestimate complaints about VDTs. Similarly, if only the most satisfied VDT users
41 respond, one may seriously underestimate complaints about VDTs. Differences between respondents and nonrespondents in VDT surveys need to be examined, and those differences need to be considered when drawing conclusions from the respondent sample. For example, if one found that younger employees were less likely to respond, one could perform analyses among the remaining respondents in an imputative search for any age differences that might influence the conclusions. Unanswered Questions No study we have reviewed has been adequate in meeting the above criteria for good research, and most of the studies have been flawed in several respects. The relationship between the use of VDTs and well-being has yet to be studied in a satisfactory, scientific manner; however, many questions are suggested--if not answeredby the published literature to date. The following is an illustrative, but by no means exhaustive, set of such questions: 1. To what extent are worker complaints (of eyestrain, backaches, emotional strains, etc.) due to pressures to perform or to the degree of worker control over performance? Are those pressures greater for workers who use VDTs compared with those who do not or for those who use them more? Does the use of VDTs introduce a unique interaction of high pressure with low control, adding further to the complaints? 2. To what degree are job stressors, emotional strains, dissatisfaction, and health-related complaints related? Are the relationships stronger or weaker in jobs involving at least some VDT use compared with those that have none? Are the strengths of the relationships different for either the extent or the kinds of VDT use? 3. If various job demands were otherwise equal, would reported complaints be correlated with the level of VDT use ? 4. What physical conditions are associated with complaints of workers using VDTs? Are those conditions unique to such workers, or would the same conditions be related to similar complaints if workers were not using VDTs? How much can workers adapt to such conditions? 5e What psychosocial factors are particularly related to the introduction of or the use of VDTs? To what extent are reported worker complaints attributable to VDTs, relative to these correlated factors? 6. Are complaints such as ocular discomfort related to the hours per day of VDT viewing? If so, is the relationship changed by introducing rest periods? If so, should the rest be total visual
42 rest or a change in visual activity? How frequent, and how long, should such rest periods be? RESEARCH DESIGN CONSIDERATIONS A number of criteria should be applied in designing future field or experimental research on effects of working with VDTs (priorities for research are discussed in Chapter 10~. It is reasonable, for example, to expect investigators to use multivariate analyses to identify the relative contributions of competing predictors (com- peting explanations) of symptoms related to vision or to psycho- social demands of work. It is reasonable, on the basis of existing research, to expect investigators to measure a variety of physical parameters of the work setting for every employee, as well as a variety of psychosocial and organizational parameters, so that contributions of each parameter can be examined while static tically controlling the effects of the others. It is reasonable to expect investigators to view VDT work as a continuum rather than as a dichotomy. Rather than artificially dividing employees into VDT and non-VDT groups, investigators should instead make use of the rich range of variance in VDT work that may occur even within a particular VDT group. control and Choice in Studies of VDT Physical Parameters One particular area that merits attention in studies of the physical parameters of VDT work concerns choice. A VDT operator may be exposed to physical condition A (such as a high-contrast display), B (moderate contrast), or C (low contrast), and inferences are drawn about which of these conditions do and do not produce strain. Such a design overlooks the condition of choice; the operator is always assigned to a condition. If, however, individual differences in preference for variety of stimuli are important, then the effect of each physical condition will vary considerably between Individuals. Suppose that one routinely includes a condition in which some participants are allowed choice; that is, subjects can alter the stimulus to their own subjective tastes (vary the independent variable). It is possible that the mere opportunity to exhibit choice will reduce some psychological strains and complaints (see Brehm, 1966~. This could be demonstrated by what is called a yoked design. In such a design, the subject in the choice condition varies the stimulus, and this variation is carefully recorded and used to present a schedule of stimulus changes to another sub- ject. The second subject receives the same variety of stimulus
43 change but lacks choice or control. Care would need to be taken to match such subjects in order to demonstrate that the effects of choice were more than the effects of allowing a person to choose the most physiologically fitting stimulus condition. Practical Considerations We do not mean to suggest that field studies must be of a perfect design to be of value; designing research always involves compro- mises, and probably no study will ever be done that will meet all of the criteria for an ideal field study. Many practical difficulties have been and will be encountered in planning and conducting field research on complaints involving VDT use. For example, it is not always possible for an investigator to use rigorous sampling techniques. Often an investigator is allowed into an institution to do research and simply told what population is available for study. Even in that situation, however, it is reasonable to expect an investigator to take the limitation into account in designing the study and analyzing the data. The use of appropriate multivariate analyses to control for the effects of extraneous variables can be particularly difficult in field research. VDT operators work with- in a complex system in which many varibles interact, probably in complex ways, to affect their well-being. While the use of multi- variate techniques is essential to understanding the interplay among the variables, the selection of which varibles and which interactions between variables to study can be problematic because there are scores of such variables and possible inter- actions. We do not yet have sufficient knowledge about which variables are important and how they may interact. Another problem involved in the use of multivariate analyses is that they require an investigator to have complete data on each questionnaire item for each subject in the population under study; response rates to questionnaire items are, however, difficult to predict in field research, and what may appear in the design phase to be a substantial population for study may in fact become small in the analysis stage when there are missing data. Absenteeism and job turnover make missing data an especially difficult problem in longitudinal studies. What is needed at this stage are some studies of samples of convenience that at least attempt to apply variety and detail of measures and in~epth techniques of data analysis. Once such studies are completed, the value of random sample designs (which may be valuable in drawing inferences about the total population of VDT users) can be more reasonably assessed.
