3
Final Status of Data Collection
The current study of the MFUA cohort of former POWs and veteran controls differs substantially from earlier efforts in that it collected data by examination rather than by mail questionnaire or record review. Response rates for the examination were disappointingly low—much lower than response rates for earlier follow-up efforts—which calls into question the representativeness of the participating group. This chapter examines in detail invitation rates, response rates, and the issue of representativeness.
Table 3.1 shows invitation and response rates by group for POWs and controls. The first appreciable difference between POWs and their corresponding controls is in their invitation rates. Despite the fact that addresses were sought from a number of independent sources, such as the Internal Revenue Service, the VA, and a commercial tracing firm (see Chapter 2 for details), address-finding was much more successful for the study's POWs than for its controls. In particular, invitations were issued to 95% or more of POWs (the PWEM group was the exception at 86%) versus roughly 75% of controls; the WK group was somewhat higher at 87%. Most of this difference between POW and control invitation rates is attributable to the design of previous follow-up efforts. The latest completed follow-up in 1984–1985 included all POW groups but only the WK controls. Because WW II controls had not been traced since 1965, their current addresses were more difficult to obtain, and in consequence their invitation rates were lower. Moreover, a high proportion of former POWs (as well as the WK controls) receive VA disability compensation and are more likely to be successfully traced using VA records.
TABLE 3.1 Invitation and Response Rates (numbers and percentages a) for the Medical Examination, by Study Group
Item |
PWP |
PWE |
PWEM |
PWK |
WP |
WE |
WK |
Number of eligible subjects at start of study |
670 (100.0) |
382 (100.0) |
258 (100.0) |
851 (100.0) |
737 (100.0) |
383 (100.0) |
861 (100.0) |
Number of invitations mailed |
650 (97.0) |
367 (96.1) |
222 (86.0) |
805 (94.6) |
551 (74.8) |
302 (78.9) |
751 (87.2) |
Responses to mailing |
|
|
|
|
|
|
|
Agreed |
210 (32.3) |
153 (41.6) |
76 (34.2) |
340 (42.2) |
145 (26.3) |
69 (22.8) |
294 (39.1) |
Refused |
132 (20.3) |
83 (22.7) |
43 (19.4) |
123 (15.3) |
110 (20.0) |
68 (22.5) |
160 (21.3) |
No response |
88 (13.5) |
53 (14.4) |
43 (19.4) |
149 (18.5) |
147 (26.7) |
85 (28.1) |
212 (28.2) |
Previous exam |
92 (14.2) |
20 (5.4) |
19 (8.6) |
126 (15.7) |
— |
— |
— |
Deadb |
128 (19.7) |
58 (15.8) |
41 (18.5) |
67 (8.3) |
149 (27.0) |
80 (26.5) |
85 (11.3) |
Number of exams completedb |
250 (38.5) |
142 (38.7) |
83 (37.4) |
408 (50.7) |
54 (9.8) |
27 (8.9) |
103 (13.7) |
PWP, prisoner of war, Pacific theater, WW II; PWE, prisoner of war, European theater, WW II; PWEM, prisoner of war, malnourished, European theater, WW II; PWK, prisoner of war, Korean conflict; WP, war veteran, Pacific theater, WW II; WE, war veteran, European theater, WW II; WK, wounded war veteran, Korean conflict. a percentages appear in parentheses. b For POWs only, subjects who died but who had provided a completed examination before their death are included in the number and percentage of completed calculations. |
TABLE 3.2 Invitation and Response Rates (numbers and percentages a) for the Psychological Questionnaire, by Study Group
Item |
PWP |
PWE |
PWEM |
PWK |
WP |
WE |
WK |
Number of eligible subjects at start of study |
670 |
382 |
258 |
851 |
737 |
383 |
861 |
Number who returned a questionnaire at exam |
42 |
34 |
16 |
99 |
24 |
10 |
43 |
Number who were mailed a questionnaireb |
316 (100.0) |
193 (100.0) |
115 (100.0) |
484 (100.0) |
270 (100.0) |
142 (100.0) |
464 (100.0) |
Responses to mailing |
|
|
|
|
|
|
|
Returned questionnaire |
130 (41.1) |
86 (44.6) |
46 (40.0) |
169 (34.9) |
71 (26.3) |
30 (21.1) |
142 (30.6) |
No response |
164 (51.9) |
92 (47.7) |
64 (55.7) |
258 (53.3) |
183 (67.8) |
104 (73.2) |
291 (62.7) |
Dead |
7 (2.3) |
3 (1.6) |
2 (1.7) |
6 (1.2) |
3 (1.1) |
2 (1.4) |
4 (0.8) |
Returned, wrong address |
15 (4.7) |
12 (6.2) |
3 (2.6) |
19 (3.9) |
13 (4.8) |
6 (4.2) |
27 (5.8) |
Total questionnaires receivedc |
172 (25.7) |
120 (31.4) |
60 (23.3) |
268 (31.5) |
95 (12.9) |
40 (10.4) |
185 (21.5) |
PWP, prisoner of war, Pacific theater, WW II; PWE, prisoner of war, European theater, WW II; PWEM, prisoner of war, malnourished, European theater, WW II; PWK, prisoner of war, Korean conflict; WP, war veteran, Pacific theater, WW II; WE, war veteran, European theater, WW II; WK, wounded war veteran, Korean conflict. a percentages appear in parentheses. b Excludes those who turned in a questionnaire at their examination, those who refused further contact, those who were identified as deceased during examination invitation, and those without a good mailing address. c Including the second mailing responses, through June 1, 1992, yields the following: PWP 211 (31.5%); PWE 144 (37.7%); PWEM 75 (29.1%); PWK 317 (37.3%); WP 114 (15.5%): WE 49 (12.8%); and WK 222 (25.8%). |
Limiting the remaining discussion to invited subjects only, one sees that POW agreement rates are higher and their refusal and no-response rates lower than those of controls. The interpretation of this differential response, however, is complicated by the fact that all POW groups included so-called "volunteers" (see Chapter 2), that is, those POWs who had come in for examination before the start of MFUA's follow-up. These POWs thus may have provided information to the study even if they subsequently refused an invitation for a reexamination or died. It must also be emphasized that the death information reported in Table 3.1 was obtained mainly as a result of unverified notification of deaths by next of kin; these data cannot be considered accurate or complete enough to permit the calculation of even crude mortality rates.
The rates of completed examinations, shown on the bottom of Table 3.1, are the most important figures in the table. They show that examination data were available for 38–50% of eligible (invited) POWs, a rate substantially higher than that of controls, of whom only 8–14% provided examination data. Such a disparity raises the obvious issue of potential nonresponse bias, which is addressed later in this chapter. However, if subjects who died were eliminated as ineligible from the control groups—a reasonable action because deceased controls, unlike deceased POW volunteers, could not possibly have provided any examination information—this marginally raises the proportion of completed examinations to 11.3% for WE, 16.1% for WK, and 13.3% for WP.
As mentioned in Chapter 2, supplemental funding from the National Institute of Mental Health enabled a last-minute collection of psychiatric data by mail questionnaire. The process of contacting study subjects directly and having them mail in their psychological questionnaires, rather than waiting for these data to come in with the medical examination, resulted in a notably higher number of responses. Questionnaires that arrived in response to the second mailing, however, came too late to be included in the analyses for this report. Hence, the response rates seen in Table 3.2 reflect only responses to the first mailing.
Table 3.2 shows that POW response rates to the questionnaire were between 35% and 40%, higher than the 20–30% response rates for controls. Adding in those psychological questionnaires that were returned at the time of the exam yields total questionnaire response rates of 26–32% for POWs and 10–28% for controls. It is noteworthy that WK had the highest total response rate of all control groups, even higher than PWP.
The low response rates observed for both POWs and controls mean that, no matter how carefully the original sample was constructed, there can be no confidence that the group of respondents accurately reflects the composition of the entire group of POWs and controls. To be fair, low response rates do not, in themselves, prove that the group of respondents is nonrepre-
TABLE 3.3 Demographic Characteristics Before Capturea for Examination Respondents and All Eligible Subjects, by Study Group (percentages in parenthesesb)
|
PWP |
PWE |
PWK |
WP |
WE |
WK |
||||||
Demographic Characteristic |
Exam |
No Exam |
Exam |
No Exam |
Exam |
No Exam |
Exam |
No Exam |
Exam |
No Exam |
Exam |
No Exam |
Year of Birth |
||||||||||||
<1910 |
8 (3.2) |
29 (6.9) |
3 (2.1) |
5 (2.1) |
1 (0.2) |
1 (0.2) |
0 (0.0) |
29 (4.2) |
1 (3.7) |
11 (3.1) |
0 (0.0) |
3 (0.4) |
1910–1919 |
147 (58.8) |
257 (61.2) |
56 (39.4) |
98 (40.8) |
13 (3.2) |
36 (8.1) |
36 (66.7) |
448 (65.6) |
12 (44.4) |
162 (45.1) |
8 (7.8) |
44 (5.8) |
1920–1929 |
95 (38.0) |
134 (31.9) |
83 (58.5) |
137 (57.1) |
203 (49.8) |
202 (45.6) |
17 (31.5) |
207 (30.3) |
12 (44.4) |
185 (51.5) |
51 (49.5) |
374 (49.3) |
1930– |
— — |
— — |
— — |
— — |
191 (46.8) |
204 (46.0) |
— — |
— — |
— — |
— — |
43 (41.7) |
338 (44.5) |
Race |
||||||||||||
White |
249 (99.6) |
419 (99.8) |
141 (99.3) |
240 (100.0) |
356 (87.3) |
376 (84.9) |
54 (100.0) |
680 (99.6) |
24 (88.9) |
358 (99.7) |
93 (90.3) |
661 (87.1) |
Black |
— — |
— — |
— — |
— — |
33 (8.1) |
48 (10.8) |
— — |
— — |
— — |
— — |
7 (6.8) |
71 (9.4) |
Other |
— — |
— — |
— — |
— — |
19 (4.7) |
18 (4.1) |
— — |
— — |
— — |
— — |
2 (1.9) |
25 (3.3) |
Component |
||||||||||||
Inductee |
34 (13.6) |
79 (18.8) |
104 (73.2) |
175 (72.9) |
114 (27.0) |
124 (28.0) |
19 (35.2) |
244 (35.7) |
11 (40.7) |
256 (71.3) |
30 (29.1) |
198 (26.1) |
Other |
213 (85.2) |
339 (80.7) |
37 (26.1) |
65 (27.1) |
294 (72.1) |
319 (72.0) |
35 (64.8) |
436 (63.8) |
13 (48.1) |
102 (28.4) |
72 (69.9) |
561 (73.0) |
TABLE 3.3 Demographic Characteristics Before Capturea for Examination Respondents and All Eligible Subjects, by Study Group (percentages in parenthesesb)
|
PWP |
PWE |
PWK |
WP |
WE |
WK |
||||||
Demographic Characteristic |
Exam |
No Exam |
Exam |
No Exam |
Exam |
No Exam |
Exam |
No Exam |
Exam |
No Exam |
Exam |
No Exam |
Marital status |
||||||||||||
Single |
231 (92.4) |
391 (93.1) |
118 (83.1) |
187 (77.9) |
348 (85.3) |
377 (85.1) |
44 (81.5) |
615 (90.0) |
23 (85.2) |
283 (78.8) |
85 (82.5) |
634 (83.5) |
Other |
18 (7.2) |
27 (6.4) |
24 (16.9) |
53 (22.1) |
59 (14.5) |
62 (14.0) |
9 (16.7) |
63 (9.2) |
1 (3.7) |
74 (20.6) |
17 (16.5) |
121 (15.9) |
Years of education |
||||||||||||
0–8 |
38 (15.2) |
100 (23.8) |
29 (20.4) |
48 (20.0) |
149 (36.5) |
163 (36.8) |
10 (18.5) |
166 (24.3) |
4 (14.8) |
92 (25.6) |
38 (36.9) |
252 (33.2) |
9–11 |
70 (28.0) |
134 (31.9) |
29 (20.4) |
69 (28.8) |
154 (37.7) |
201 (45.4) |
12 (22.2) |
212 (31.0) |
4 (14.8) |
83 (23.1) |
31 (30.1) |
326 (43.0) |
12 |
87 (34.8) |
118 (28.1) |
54 (38.0) |
88 (36.7) |
84 (20.6) |
59 (13.3) |
20 (37.0) |
195 (28.6) |
8 (29.6) |
107 (29.8) |
25 (24.3) |
131 (17.3) |
>13 |
54 (21.6) |
69 (16.4) |
30 (21.1) |
34 (14.2) |
21 (5.1) |
17 (3.8) |
11 (20.4) |
106 (15.5) |
8 (29.7) |
75 (20.9) |
8 (7.8) |
49 (6.5) |
Arm or service at selection |
||||||||||||
Arms |
115 (46.0) |
226 (53.8) |
57 (40.1) |
130 (54.2) |
363 (89.0) |
402 (90.7) |
33 (61.1) |
392 (57.4) |
10 (37.0) |
214 (59.6) |
93 (90.3) |
705 (92.9) |
Technical |
39 (15.6) |
82 (19.5) |
12 (8.5) |
25 (10.4) |
44 (10.8) |
41 (9.3) |
5 (9.3) |
108 (15.8) |
0 (0.0) |
1 (0.3) |
9 (8.7) |
51 (6.7) |
Air Corps |
96 (38.4) |
111 (26.4) |
73 (51.4) |
85 (35.4) |
— — |
— — |
16 (29.6) |
180 (26.4) |
14 (51.9) |
143 (39.8) |
— — |
— — |
Totalc |
420 |
250 |
240 |
142 |
443 |
408 |
683 |
54 |
359 |
27 |
759 |
103 |
PWP, prisoner of war, Pacific theater, WW II; PWE, prisoner of war, European theater, WW II; PWK, prisoner of war, Korean conflict; WP, war veteran, Pacific theater, WW II; WE, war veteran, European theater, WW II; WK, wounded war veteran, Korean conflict. a Control date equivalent to POW capture date. b Percentages appear in parentheses. c The counts for subgroups may not match the totals because unknowns are omitted. |
sentative. However, the statistical principles that determine how inferences may be drawn from statistical samples do not apply when response rates are as low as they are in this instance. Consequently, a case for the representativeness of the group of respondents simply cannot be made.
Nevertheless, it is still worthwhile to examine the actual composition of the group of respondents and compare it with that of the original sample to determine how unrepresentative it appears to be. There are two kinds of data available for comparison: demographic and hospitalization data.
Demographic data were gathered from military records and are available from virtually the entire sample. Table 3.3 shows demographic data by POW and control study group, comparing those having completed examinations with all subjects eligible for the study. There were relatively few differences between subjects with completed examinations and all eligible subjects. Among WW II POWs, there were appreciably higher proportions of men with service in the Army Air Corps—servicemen who were captured when their planes were shot down were held as POWs; among the PWP and PWK groups, respondents had notably higher levels of education. In the WE group, compared with other controls, there was a higher proportion of examined men who were single at the time of entry into service or who were inductees. The usual kinds of statistical tests of differences are not appropriate here, because such tests focus on the effects of sampling error (i.e., the specific error that would arise from having examined only a sample and not the whole group of either POWs or controls); with such low response rates, there is the clear potential for large, unmeasurable errors due to nonresponse.
Not all demographic characteristics of possible interest were, or could be, examined. Factors such as the distance of a subject from the nearest VA medical facility, for example, were applicable only to those who came to the examination and were reimbursed for travel expenses by MFUA. Of course, not even a complete examination of all available demographic data and a finding of few or no differences would provide appropriate, sufficient evidence to conclude that a group of respondents is a representative sample. Nevertheless, based on an examination of these selected demographic characteristics, there is little evidence to suggest that the group of respondents differs substantially from nonrespondents.
The comparison hospital data also provide additional evidence that responders and nonresponders are not radically different. Prior to the beginning of the examination study, data were obtained on VA hospitalizations from 1969 to 1985 for all of the study's eligible subjects. Table 3.4 shows VA hospitalization rates for those with completed exams, for eligible nonrespondents, and for ineligible subjects. (The latter are those to whom invitations were not mailed or who were discovered to be deceased.)
The most important comparison in Table 3.4 is between subjects with
completed exams (respondents) and those eligible to be examined who did not provide exam data (nonrespondents). In most cases, the two VA hospitalization rates are fairly similar across the POW and control groups. There are some differences—particularly in the Pacific theater WW II groups—but interestingly the direction of these differences is not uniform across all study groups. Subjects with completed examinations had higher VA hospitalization rates in the PWE, PWEM, WP, and WK groups; subjects with completed exams in the WE, PWP, and PWK groups had lower VA hospitalization rates. With the exception of the WP group, ineligible subjects had the highest VA hospitalization rates, presumably because that group included deceased subjects who might be expected to have higher VA hospitalization rates.
These findings are further concrete, if only partial, evidence that differences identified in the health of respondents and nonrespondents were relatively small; indeed, similar evidence was adduced by Beebe (1975) in his 1967 morbidity follow-up study. Along the same lines, Page (1991) estimated the effects of nonresponse in the 1984–1985 questionnaire follow-up and found no evidence of any appreciable bias. Sample sizes in Table 3.4 are nonetheless quite small, especially in the control groups, and the WP group is unusual in the large difference in hospitalization rates between respondents and nonrespondents and in the fact that respondents had higher
TABLE 3.4 Comparison of VA Hospitalization Rates for Examination Study Respondents and Nonrespondents, by Study Group (average ± standard error, N in parentheses)
|
Type of Respondent |
||
Study Group |
Exam Completed |
Eligible, Exam Not Completed |
Not Eligiblea |
PWE |
0.338 ± 0.088 (142) |
0.244 ± 0.067 (176) |
0.547 ± 0.167 (64) |
PWEM |
0.627 ± 0.180 (83) |
0.421 ± 0.118 (107) |
0.705 ± 0.203 (68) |
WE |
0.160 ± 0.095 (25) |
0.228 ± 0.069 (197) |
0.783 ± 0.147 (161) |
PWP |
0.892 ± 0.105 (250) |
1.058 ± 0.111 (293) |
1.520 ± 0.201 (127) |
WP |
0.778 ± 0.230 (54) |
0.342 ± 0.060 (348) |
0.411 ± 0.073 (335) |
PWK |
0.745 ± 0.074 (408) |
0.799 ± 0.093 (338) |
1.143 ± 0.209 (105) |
WK |
0.422 ± 0.119(102) |
0.387 ± 0.047 (564) |
0.610 ± 0.112(195) |
PWP, prisoner of war. Pacific theater, WW II; PWE, prisoner of war, European theater, WW II; PWEM, prisoner of war. malnourished, European theater, WW II; PWK, prisoner of war, Korean conflict; WP, war veteran, Pacific theater, WW II; WE, war veteran, European theater, WW II; WK, wounded war veteran, Korean conflict. a Unverified death, invitation never mailed, or invitation returned because of wrong address. |
VA hospitalization rates than ineligibles. While not entirely comforting, the data in Table 3.4 do suggest that nonresponse bias is not overwhelmingly large, although it cannot be asserted that VA hospitalization rates are the most appropriate way to gauge overall health.
Because the low response rates for POWs and controls vitiate any arguments about the representativeness of the groups of POW or control examinees, the examinations that were collected must be considered case study material and the analyses in the remainder of this report purely descriptive. With this understanding, the report moves next to further discussion in Chapter 4 of the examination format and details of data collection, along with the presentation of some preliminary results and comparisons with the results of another VA exam-based study. After this, the principal study questions will be taken up in Chapter 5.
REFERENCES
Beebe GW. 1975. Follow-up studies of World War II and Korean war prisoners. II. Morbidity, disability, and maladjustments. Am. J. Epidemiol. 101:400–422.
Page WF. 1991. Using longitudinal data to estimate nonresponse bias. Soc. Psychiatry Psychiatr. Epidemiol. 26:127–131.