2
Data Collection Procedures
The current MFUA study is the sixth such study of the group of POWs and controls assembled some four decades ago. Previous efforts have focused on mortality, as ascertained from death certificates, and on morbidity, as reported in VA records and in questionnaires mailed to study subjects. The data for the current study are unique: both former POWs and veteran controls were invited to undergo a comprehensive medical and mental health examination at a nearby VA medical facility; the results of the examination were sent to MFUA for analysis.
This approach took advantage of the ongoing POW examination program, called the POW Protocol Examination program, that was already in place in VA medical facilities when this research began. Because of the existing program, MFUA collected its research examination data by inviting study subjects to participate in the VA program and then receiving a copy of the VA examination results. In order to include the MFUA research subjects in the VA program, it was necessary for the VA Central Office to issue a circular to its field stations, directing them to perform these examinations and send copies to MFUA. Because of administrative procedures, there were at one time two VA circulars in the field (one for POW examinations and one for control examinations); copies of these circulars can be found in Appendix A.
Although the overall study design was quite straightforward, a complicating factor arose in the form of potential study subjects who had already been examined under the VA program for their own personal reasons. Originally,
these subjects, called ''volunteers,'' were not asked to undergo a reexamination but instead were identified by matching them against the VA POW exam tracking system, which contains a record for each POW who has undergone the VA's protocol exam. Requests for their already completed examinations were then sent directly to the VA field stations. (More details about this process are provided later in this chapter.)
The issue of volunteers became somewhat more complicated, however, when a site visit by a VA team in August 1989 resulted in a recommendation that volunteers be reexamined. As a result, volunteers were mailed invitations and processed similarly to study subjects who had not been previously examined. The only remaining complication was the possibility that MFUA might receive reports of two examinations for an individual—one done before its invitation and one done afterward. In such cases, only the most recent exam was included in the analyses.
If a study subject had not undergone a POW protocol examination (or later, after volunteers were solicited, even if he had), an invitation was issued by MFUA for the man to undergo examination at a nearby VA hospital. Using addresses on file from previous MFUA follow-up studies or addresses obtained from the VA, the Internal Revenue Service, or a commercial tracing firm (Equifax), MFUA staff mailed subjects as many as three invitations, each of them tracked by a computerized mailing system. If a new address was obtained at any time during the study, fourth, fifth, and sixth mailings were attempted if needed.
Once a study subject agreed to be examined, this fact was logged into the mailing system and used to produce a list of subjects to be scheduled for examination at each VA hospital. Over the course of the study, more than a dozen such schedule lists were sent to VA medical facilities. When the examinations were completed, copies were sent to MFUA, which were then sent out to be abstracted, coded, and computerized by trained nosologists under MFUA's contract with GRG Associates, a local subcontractor.
DETAILS OF THE DATA COLLECTION PROCESS
Much of the invitation letter process was automated, and a computerized mail system developed in an earlier POW study was used to generate and print mailing labels and keep track of mailing dates and mail status information. The complete mail package included a cover letter, a postage-paid envelope, and a response form on which the subject indicated his willingness to participate and provided contacting information, such as his home or work telephone number (see Appendix A).
When responses were received from the mailing, they were logged into the mailing system. In most cases the response (or lack of it) was easy to categorize, and the following codes were simply entered into the system:
1 = invitation returned, agreed to participate
2 = RWA (returned without address)
3 = refusal, do not mail again
4 = invitation returned marked deceased, death not verified
5 = deceased, verified; do not mail again
6 = unable to participate
7 = previously examined
A = subject claims he was already examined
R = recontact veteran to clarify exam status
V = volunteer (this code was only assigned before mailing).
Some discussion is necessary regarding a few of these codes. An unverified death (code 4) was one for which MFUA received notice but no official record, such as a death certificate. "Unable to participate" (code 6) was differentiated from refusal by the subject's indicating a willingness to be examined but his also citing conditions such as poor health or long distances that made his participation difficult or impossible. Code A was assigned when a subject claimed a previous examination even though he had not been identified as a volunteer. In such instances, MFUA attempted to contact the VA hospital to obtain a copy of this exam. If no examination was available, the mail system code was changed to R (recontact), and another letter was sent. Code R was also used in cases in which, for example, the subject returned the response form but failed to check the box indicating his willingness to participate. Any address changes noted on the response form were also logged into the mail system upon return.
At the same time that examination requests were being sent to individuals, requests were being sent to VA hospitals for copies of already completed exams undergone by the "volunteers." To accomplish this, lists of subjects who were thought to have undergone examination were sent to VA medical facilities. In this request for information, as with the previously described schedule list, two identical copies of the letter and list were sent to the facilities from which completed exams were requested: one copy went to the POW administrative coordinator and one to the physician coordinator. (Every VA medical facility that performs POW examinations has both a physician coordinator and an administrative coordinator.) Requests for completed examinations were made periodically.
After a subject had indicated an interest in being examined, it was necessary to notify a nearby VA medical center of his interest. Each such subject was first assigned to the nearest VA medical facility, gauging distances by road atlas. There was also an attempt to assign the subject to a VA hospital based on VA-defined catchment areas, but there were some cases in which the VA facility determined by catchment area would not have been the one closest to the subject. In other cases, the respondent may have indicated a preferred VA facility, which was almost always honored.
Once the hospital assignment had been made, the schedule lists were produced automatically at specified times, each containing the names and identifying numbers of the study subjects to be examined. Transmitted with the schedule list was the original response form for each person on the list, accompanied by a cover letter. The computerized mail system tracked the status of each exam request by individual subject.
Before contacting a subject for examination, however, each VA hospital involved in the study was required to make arrangements for review of the study protocol by the appropriate local review committee that dealt with human subjects research. Although a study protocol and sample informed consent document were included in the VA circulars guiding the study, each hospital was responsible for securing the approval of its local committee before undertaking data collection.
When the completed examinations were returned, they were first carefully separated, checked, and logged in. The complete examination package was supposed to include all of the following: a POW medical history form (VA Form 10-0048); a summary sheet containing the final list of diagnoses, prepared by the POW physician after all laboratory results and consultations were received; a physician's history and physical; a social work consultation (sometimes called a psychosocial history); a mental health evaluation (either a psychological or psychiatric consultation), which was supposed to include the posttraumatic stress disorder (PTSD) portion of the Structured Clinical Interview for the third revised edition of the Diagnostic and Statistical Manual (DSM-III-R); and finally the four psychological questionnaires, known as forms A (the Beck depression scale), B (the Center for Epidemiologic Studies depression [CES-D] scale), C (the 90-item Hopkins symptom checklist [SCL-90]), and D (the Mississippi PTSD scale). Incomplete examinations were noted in a separate file, and attempts were made later to contact the VA hospitals and obtain these missing pieces.
Because the research exams included additional items such as the structured clinical interview for PTSD and the psychological questionnaires, a brief discussion of these items is warranted. An abbreviated version of the Structured Clinical Interview for DSM-III-R, Non-patient Version (Spitzer et al., 1986), hereafter SCID, was used in this study because it was felt that the full-length SCID—some 90 pages long—could not be successfully administered. Given the potential importance of PTSD among POWs and the need to keep short any additional data collection demands, only the 8-page PTSD portion of the SCID was chosen for inclusion in the exam; in order to use this form, references to Vietnam service were removed to make it applicable to POWs and controls from WW II and the Korean conflict. Although no special training was given to field personnel regarding the SCID, instructions concerning its administration were included in the directives (i.e., VA circulars) sent to all VA facilities; there were no explicit attempts to assess inter-
rater reliability. The PTSD portion of the SCID is designed in such a way as to allow the interviewer independently to record information pertinent to both lifetime and current symptoms of PTSD. The interviewer then provides his or her separate assessments of these diagnoses. In general, the same person who performed the mental status examination also administered the SCID.
The four psychological questionnaires were standard instruments, each of which was chosen for different purposes. The CES-D (Radloff, 1977) was an obvious choice, given its use in the 1984–1985 follow-up, and offered the additional advantage of obtaining test-retest data. The Beck depression scale (Beck et al., 1961) was chosen because it is more widely known and accepted in clinical circles than the CES-D, which has been used more in epidemiologic and general population studies. The Mississippi PTSD scale (Keane et al., 1984) is a paper-and-pencil test originally developed to measure the presence of PTSD in populations of Vietnam veterans. The SCL-90 (Derogatis et al., 1973) is a widely used general checklist of psychological symptoms. In all cases, standard scoring of these instruments was done, and well-established "cut-off" scores were used: for the CES-D scale, a score of 16 or more being evidence of depressive symptoms; for the Beck scale, less than 10 indicating no evidence of depression; 10–18, evidence of mild depression; 19–29, evidence of moderate depression; and 30 or more, evidence of marked depression. For the 35-item Mississippi scale, 89 or more was evidence of PTSD (Kulka et al., 1991).
The status of every exam received was tracked by the computerized mail system. Complete and incomplete exams were simply coded C and I, respectively, but in some instances other status codes were assigned based on information received from the VA hospital if, for example, the exam itself could not be completed. The following codes were provided only by VA hospitals on returned schedule lists and were entered directly into the exam status field:
1 = exam scheduled
2 = exam previously done—do not repeat if exam was complete
3 = VA unable to contact
4 = subject refused examination (at the VA hospital)
5 = subject physically unable to come for exam
6 = subject deceased (reported by VA hospital)
7 = VA hospital able to contact but subject moved out of area.
Three additional codes—U, N, and G—were infrequently entered. Code U was assigned when the VA hospital was unable to locate an examination that had supposedly been completed; code N was given when there was no exam done on a volunteer; and code G was used when the medical facility suggested that MFUA request the subject's VA claims folder (a "form G" request) to obtain a copy of the exam.
In some cases, MFUA found that an exam had been transferred from one VA hospital to another, which usually happened when a veteran found that another VA facility was more convenient than the one MFUA had assigned. The computerized mail system, which allowed up to three VA medical facilities to be entered onto the file, was used to track such transfers. The contract with the VA did not provide for payment of research subjects for their participation in the study. It did, however, allow MFUA to reimburse participants directly for their mileage expenses at a slightly higher rate than permitted by local VA facilities.
The abstracting, coding, and computerization of completed examinations was done by a subcontractor, GRG Associates. Examinations were generally sent in batches of 50, and each batch of completed exams was returned along with a floppy disk containing the abstracted information and a hard-copy printout of the abstracted and coded information. All medical conditions noted in the examination were coded by a trained nosologist using the ninth edition of the International Classification of Diseases, Clinical Modification (ICD-9-CM).
Each coded condition was also categorized as to its source within the examination: discharge summary; history; findings; laboratory, x-ray, or other such services; psychiatric or psychological consultation; medical consultation; or VA benefits application form. Thus, a specific condition might appear several times, with different sources, in a single coded record. Data on the resolution of each medical condition were also provided by the coder, based on available material in the exam (e.g., whether the subject was still under treatment, still receiving medication, no longer had a problem, etc.). If there was no information about a condition being resolved, it was considered unresolved. Data on medications were also recorded exactly as noted in the record, and each recorded medication was associated with a particular diagnosis; these recorded data, however, were not edited or abstracted further and so are not included in the analyses of this report. In each batch, a sample of hard-copy abstracts was spot-checked against the original examinations. If an unusually high number of errors were detected, that batch was sent back to GRG for correction.
The POW medical history forms (VA Forms 10-0048) were keyed by MFUA personnel into a separate data file, as were the SCID and psychological test forms. A random sample of medical history forms was printed and checked by hand against the original forms.
Toward the end of the study, it became clear that the number of completed exams would be much greater than the number of completed psychological questionnaires, in part because volunteers who were not reexamined did not get a chance to fill out the psychological questionnaires. It was decided to address a last-minute, separate direct mailing to study subjects to request the psychological questionnaire data. With the sponsorship of the
National Institute of Mental Health, MFUA mailed some 2,000 questionnaires to POWs and controls who were not known to be dead, who had a valid mailing address, and who had no psychological questionnaires on file. When returned, these questionnaires were keyed and added to the data file for analysis. A second mailing was made to nonrespondents, but these data were available too late for inclusion in this report.
The next chapter reviews the final status of data collection, with a focus on the response rates for the medical examination and for the psychological questionnaires.
REFERENCES
Beck AT, Ward CH, Mendelson M, et al. 1961. An inventory for measuring depression. Arch. Gen. Psychiatr. 4:561–571.
Derogatis LR, Lipman RS, Covi L. 1973. The SCL-90: an outpatient psychiatric rating scale. Psychopharmacol. Bull. 9:13–28.
Keane TM, Malloy PF, Fairbank JA. 1984. Empirical development of an MMPI subscale for the assessment of combat-related posttraumatic stress disorder. J. Consult. Clin. Psychol. 52:888–891.
Kulka RA, Schlenger WE, Fairbank JA, et al. 1991. Assessment of posttraumatic stress disorder in the community: prospects and pitfalls from recent studies of Vietnam veterans. Psychol. Assess. 3:547–560.
Radloff LS. 1977. The CES-D scale: a self-report depression scale for research in the general population. Appl. Psychol. Meas. I:385–401.
Spitzer RL, William JB, Gibbon M. 1986. Structured Clinical Interview for DSM-III, Non-patient version (SCID-NP-V; 12/1/86). Biometrics Research Department, New York State Psychiatric Institute. New York, NY.
