Evaluating AIDS Prevention Programs Expanded Edition (1991) / Chapter Skim
Currently Skimming:
F The Use of Selection Models to Evaluate AIDS Interventions with Observational Data
F The Use of Selection Models to Evaluate AIDS Interventions with Observational Data
Pages 342-364
The Chapter Skim interface presents what we've algorithmically identified as the most significant single chunk of text within every page in the chapter.
Select key terms on the right to highlight them within pages of the chapter.
From page 342... ...
It is this outcome that will be Me focus of the analysis here. This paper was presented at the Conference on Nonexperimental Approaches to Evaluating AIDS Prevention Programs convened on January 12-13, 1990 by the Panel on the Evaluation of AIDS Interventions of the National Research Council.
|
From page 343... ...
The econometric literature on program evaluation underwent a major alteration in its formal framework after the separate development of "selectivity bias" techniques in the mid-1970s. Originally, the selectivity bias issue in economics concerned a miss~ng-data problem that arises In the study of individual labor market behavior, namely, the inherent
|
From page 344... ...
The recent work of Heckman and Robb (1985a, 1985b) represents the most complete and general statement of the selectivity bias problem In program evaluation using observational data, and provides He most Borough analysis of the conditions under which the methods win yield good estimates and of the estimation me~ods available to obtain such estimates.
|
From page 345... ...
. Ideally, we wish to know the level of Yip for such individuals we would like to know what their level of prevention behaviors, for example, would have been had they not gone through the program.
|
From page 346... ...
The equation will fall to hold under many plausible circumstances. For example, if those who go through a C&T program are concerned with their health and have already begun adopting prevention behaviors even before entering the program, they wiD be quite different from those who do not go through He program even prior to receiving any program services.
|
From page 347... ...
has no direct relationship to Yip (e.g., no direct relationship to individual prevention behavior)
|
From page 348... ...
. For example, if Yip follows a normal, logistic, or some over distribution with a finite set of parameters, identification of a program effect free of selection bias 5 For example, if Y1 is the mean value of the outcome variable in the city with the program and Yo is the mean value in the city without one, and if p is the fraction of the relevant subpopulation in the first city that participated in the program, the impact estimate would be calculated as (Ye—YO)
|
From page 349... ...
However, this method will not be especially useful for the AIDS interventions because very little is known about the distribution of sexual prevention behaviors, for example, in the total population or even the high-risk population. Consequently, this method will not be considered further.
|
From page 350... ...
as would be the case, for example, if those who later undergo C&T have higher prevention behaviors in the first place.
|
From page 351... ...
C to C, which is a larger rate of growth Han for non-par~cipants. The estimate of the treatment effect, cr.
|
From page 352... ...
or the sexual behavior histories of C&T participants and non-participants, for example, stands in contrast to the situation faced when conducting a randomized tIia] where, strictly speaking, only a single post-treatment cross section is required.
|
From page 353... ...
More periods of data make possible estimates of treatment effects which are equal to Rose obtainable from a randomized trial under weaker and weaker conditions, thereby strengthening the reliability of
|
From page 354... ...
Put differently, it must be the case that it if we observe two individuals at time t-] who have exactly the same history of Yip up to that time (e.g., the exact same history of sexual prevention behaviorsWand who therefore look exactly alike to the investigators they must have the same value of YE in the next time period regardless of whether they do or do not undergo the treatment.
|
From page 355... ...
holds Data Set\ 1 _ A1 holds A2 does not hold A3 holds Data Set 2 A1 holds A2 does not hold ~ Model IV A3 does not hold Data Set 3 Model II Model A1 does not hold | A2 holds A3 holds Model V A1 does not hold A2 does not hold A3 holds FIGURE F-2 Estimable models win different data sets. Data set 1: Single post-program, no Zi.
|
From page 356... ...
. For the treaunent-effect estimate from this model to be accurate still requires the assumption Mat Be Zi is a legitimate instrument that Me differential availability of Me program across cities is not related to the basic levels of prevention behavior In each city (i.e., that Zi and Yip are independent)
|
From page 357... ...
where participants and non-participants across cities were pooled into one data set and city location was not controlled for because the vanable was not available provides a test for whether intercity variation is a legitimate Zi. If it is not (e.g., if program placement across cities is based on need~then Models ~ and m will produce quite different treatment estimates, for Model ~ does not control for city location but Mode} m does (Mode]
|
From page 358... ...
An additional pre-program data point or an additional Zi variable would ennch the data set and permit the assumptions of Me two models to be tested. New models made possible by increasing the richness of the data set permit the evaluator to discard more and more assumptions arid therefore obtain impact estimates that are more and
|
From page 359... ...
The CBO programs under consideration here are Pose which conduct local community health education and risk reduction projects. The types of programs offered are more diverse than those offered In the C&T programs, ranging from educational seminars for AIDS educators to the establishment of focus groups, conducting counseling, educating high-risk groups about risk reduction strategies, and the sponsoring of street fairs and performing arts activities In support of education and risk reduction.
|
From page 360... ...
Second, there are likely to be serious political difficulties as well, for AIDS treatment has already become a highly politicized issue in local communities, and popular resistance to randomization will no doubt be even more difficult to overcome Can it already is for other programs. Third, more than in most randomized trials, those In the AIDS context require a high degree of cooperation from the indigenous staff operating the programs, both to elicit accurate responses from the subjects, to reduce attntion, and in light of confidentiality requirements that often make it difficult for outside evaluators to be integrally involved In the operation and data collection of the experiment.
|
From page 361... ...
For example, it is extremely unlikely that arty sociodemographic or health characteristic of individuals themselves would be appropriate. Health status, education level, prior sexual history, and other such characteristics no doubt affect the probability that an individual enrolls in a C&T or CBO program but also unquestionably are independently related to prevention behavior as well.
|
From page 362... ...
For example, it has been estimated that 60 percent of the male homosexual population in San Francisco has not been tested for REV infection and has, therefore, almost certainly also not enrolled in a C&T or CBO program.~3 Why this percent is so high could be investigated. Perhaps the 60 percent who have not been tested are those with low probabilities of HIV In the first place, or those who are already practicing prevention behaviors -in this case, no appropriate Zi would be available.
|
From page 363... ...
The success of the cohort is partly a result of solid confidentiality measures as well as the heavy involvement of local gay community leaders and trained local staff from the beginning of the study. Other cohort collection efforts include the CDC cross-city study of O'ReiBy, involving both homosexual men as well as IV drug users; the study of seven CBOs headed by Vincent Mor at Brown University; He San Francisco city clinic cohort and Hepatitis B cohort; and the upcoming Westat cohort sponsored by NCHSR.
|
From page 364... ...
(1974) Comments on Selectivity Biases in Wage Comparisons.
|
Key Terms
This material may be derived from roughly machine-read images, and so is provided only to facilitate research.
More
information on Chapter Skim is available.