National Academies Press: OpenBook
« Previous: LEGAL ISSUES AND DRUG PARAPHERNALIA
Suggested Citation:"EVALUATION METHODS." Institute of Medicine and National Research Council. 1994. Proceedings--Workshop on Needle Exchange and Bleach Distribution Programs. Washington, DC: The National Academies Press. doi: 10.17226/4552.
×

EVALUATION METHODS

Suggested Citation:"EVALUATION METHODS." Institute of Medicine and National Research Council. 1994. Proceedings--Workshop on Needle Exchange and Bleach Distribution Programs. Washington, DC: The National Academies Press. doi: 10.17226/4552.
×
This page in the original is blank.
Suggested Citation:"EVALUATION METHODS." Institute of Medicine and National Research Council. 1994. Proceedings--Workshop on Needle Exchange and Bleach Distribution Programs. Washington, DC: The National Academies Press. doi: 10.17226/4552.
×

ASSESSING THE EFFICACY OF NEEDLE EXCHANGE PROGRAMS: AN EPIDEMIOLOGICAL PERSPECTIVE

NOREEN V. HARRIS and JAMES P. MCGOUGH, King County Department of Public Health, Seattle, Washington; and NOEL S. WEISS, Department of Epidemiology, University of Washington

INTRODUCTION

In order to evaluate the efficacy of needle exchange programs, we must examine what they attempt to accomplish, and how they go about accomplishing it. Needle exchange programs have as their goal reducing the transmission of HIV (and other pathogens) among injection drug users. Such programs typically attempt to achieve their aims through three different kinds of interventions: first, the provision of materials such as sterile needles, bleach for syringe cleaning, and condoms; second, by providing education and information; and third, by referring clients to other interventions, such as drug treatment programs, social services, and medical care.

The success of all of these interventions may be achieved through behavior change. Freer access to sterile needles, bleach, and condoms may offer participants the materials necessary to adopt new, less-risky injection and sexual practices, and education may provide them with the motivation and skills necessary to make those changes. Drug treatment and access to social and medical services may decrease a drug-user's need to inject drugs, and so the number of times needles are shared.

It is also possible, however, that needle exchanges may be successful at reducing the transmission of HIV independently of changes in sharing patterns, if they reduce the likelihood that the needles that are shared are contaminated with HIV. In the absence of behavior change, such a reduction would take place if exchange programs either replaced a supply of contaminated needles with sterile ones or reduced "the number of people each needle shared" by shortening the circulation time of the average needle before it is replaced by a sterile one (Kaplan 1993). If a needle exchange program succeeded only in supplanting an existing supply of sterile needles, then there would be no reduction in the likelihood of their contamination with HIV.

CAUSAL INFERENCE

The evaluation of needle exchange programs involves a quest for evidence that needle exchanges caused a particular outcome of interest.

The development of causal hypotheses relies on an inductive approach which is commonly guided by a set of criteria (Hill 1965):

  • Presence of an association, especially one that is consistent across studies.

Suggested Citation:"EVALUATION METHODS." Institute of Medicine and National Research Council. 1994. Proceedings--Workshop on Needle Exchange and Bleach Distribution Programs. Washington, DC: The National Academies Press. doi: 10.17226/4552.
×
  • Presence of a strong association (i.e., a large relative risk). Weak associations may be causal, but are more likely than strong ones to be wholly the result of non-causal factors.

  • A temporal relationship in which the putative cause precedes the effect.

  • A gradient in the strength of the association that accords with predictions based on our biological, social, or psychological understandings of the issue (Weiss 1981).

  • The plausibility of a causal relationship on other, non-epidemiological, grounds.

After the development of causal hypotheses, a deductive approach is followed in which predictions arising from those hypotheses are subjected to empirical tests. While we cannot "prove" causal hypotheses, we can (in principle) refute them. Hypotheses not falsified by the data at hand are "confirmed" in that they remain reasonably good explanations until they are falsified by new data, and are replaced by other hypotheses that better explain the observations (Popper 1965; Rothman 1986).

OUTCOMES OF INTEREST

Outcomes Related to Reducing Risk of HIV Transmission

In order to determine if needle exchange programs succeed in their aim of reducing the parenteral and sexual risk of HIV transmission among injection drug users, one could measure the prevalence or incidence of HIV, the behavioral predictors of HIV transmission, surrogate measures of those behaviors, or the number of different persons sharing individual needles.

HIV

There are two means of determining whether or not an individual has acquired HIV or any other parenterally or sexually transmitted infection during a specific time period. One could establish whether or not seroconversion occurred between baseline and follow-up testing or, when only one test is done, the presence of short-lived antibodies or antigens can in principle serve as markers of recent infection (Hart et al. 1989; van den Hoek et al. 1989; Nelson et al. 1991; Kaplan et al. 1991; van Ameijden et al. 1992).

Suggested Citation:"EVALUATION METHODS." Institute of Medicine and National Research Council. 1994. Proceedings--Workshop on Needle Exchange and Bleach Distribution Programs. Washington, DC: The National Academies Press. doi: 10.17226/4552.
×
Behavior

Some of the parenteral behaviors associated with HIV infection which could be measured include: "sharing" of injection equipment (the word here meaning injecting with needles after their use by others); sharing of injection equipment without effective cleaning; sharing with high-risk individuals; and the frequency of injection with uncleaned shared equipment (Donoghoe et al. 1989; van Ameijden et al. 1992; Guydish 1993).

Some of the sexual behaviors related to HIV infection which could be measured include: sex with multiple partners; sex without condoms; and sex with high-risk partners (Donoghoe et al. 1989; Stimson et al. 1989; Hart et al. 1989).

Surrogate Measures of HIV

Serological or other clinical evidence of recent acquisition of parenterally-transmitted or sexually-transmitted diseases other than HIV may serve to indicate that individuals were involved in high-risk activities that could also transmit HIV (van Haastrecht et al. 1991; Brettle 1991).

Number of Persons Sharing Each Syringe

Even in the absence of significant change in the propensity to share needles, a needle exchange might reduce the number of people who share the average needle. The fewer people sharing a needle, the smaller the chances of that needle becoming infected with HIV, and so the lower the risks of disease transmission. One could, in principle, measure the number of sharers directly, measure the average time that needles spend in circulation in a community (Donoghoe et al. 1989; Kaplan 1993), or measure syringe barrel wear as a surrogate of syringe use (Smith et al. 1981).

Outcomes Related to Injection Drug Use

It is possible that needle exchange programs promote drug use by "condoning" it, by making drug injection easier, or by fostering the initiation of new drug injectors. On the other hand, needle exchange programs might lead to a reduction in drug use by facilitating entry to drug treatment, or by supporting users in stopping or reducing injection. The mere presence of needle exchange programs might convince injectors of the seriousness of the HIV epidemic, and of the need to reduce or stop drug use to avoid AIDS.

Possible measures of drug use include initiation or cessation of injection, changes in the frequency of injection, and a shift to non-injection modes of drug use (U.S. General Accounting Office 1993).

Suggested Citation:"EVALUATION METHODS." Institute of Medicine and National Research Council. 1994. Proceedings--Workshop on Needle Exchange and Bleach Distribution Programs. Washington, DC: The National Academies Press. doi: 10.17226/4552.
×

POTENTIAL RELATIONSHIPS OF EXPOSURE TO OUTCOME

The impact of needle exchange might be related to the duration, recency, or directness of an individual's exposure to the exchange. Choice of a particular model or models of the possible relationship between exposure and outcome will have an important influence on the selection of appropriate measures of exposures and outcomes.

Duration Can Be Modelled in a Number of Different Ways
The One-Hit Model

Patrons of the exchange might achieve benefit after a single exposure. This model reflects what might happen if the very existence of the exchange alerted clients to the risk of AIDS, and convinced them to stop sharing needles or to stop using drugs. To assess such an impact, it would be necessary to classify participants in terms of past exchange use (ever versus never) and to measure the relevant outcomes after exposure to needle exchange (or in the cases of non-exchangers, during a comparable time period).

The Threshold Model

The effect of needle exchange might manifest itself only after a sufficient minimum number of repeated exposures. For example, it may be the case that outreach workers staffing the exchange table develop a rapport with clients, and earn their trust, only after a certain number of exchange encounters. It also might well be the case that the success of various risk reduction educational and other interventions would be dependent on the development of that rapport and trust, and so require a certain number of needle exchange visits.

The Dose-Response Model

An effect of needle exchange might be related to the total amount of exposure; the effect might increase with increasing frequency, duration, or intensity of exposure, with no threshold level of exposure.

Suggested Citation:"EVALUATION METHODS." Institute of Medicine and National Research Council. 1994. Proceedings--Workshop on Needle Exchange and Bleach Distribution Programs. Washington, DC: The National Academies Press. doi: 10.17226/4552.
×
Recency of Exchange Might Also Be Modelled in Different Ways
Latency of Effects

In some cases the effects might be immediate, while in others a specified latent period might be required before benefit or harm accrues (Schlesselman 1982). If effects were immediate, a study that compared repeat exchangers to non-exchangers would not be able to detect any differences between them if the benefit from the exchange manifested itself right after the first visit.

Transience of Effects

The effects of needle exchange might be either permanent or transient. A change to lower-risk behavior might, for example, not persist as the novelty of the needle exchange intervention wanes, leading to "relapses" to higher-risk activity.

Directness of Effects

It is possible, particularly in communities with large-volume needle exchanges, that indirect effects might be of importance. Non-exchangers might derive "herd immunity" from needles and education provided them by those who are exchangers. In such circumstances, observational studies of individuals may not be able to tease out the effects of needle exchange without collecting data on sources of needles and information among both needle exchangers and non-exchangers. In this case it may also be more difficult to demonstrate an association between needle exchange and HIV risk. Direct and indirect effects might be in operation at the same time, of course, and they may have different impacts on different outcomes. As far as we know, all studies of individuals have looked only at direct effects.

STUDY DESIGN

There have been numerous attempts to glean information on the effect of needle exchange programs from surveys that do not include comparable, concurrently sampled, control groups (Donoghoe et al. 1989; Kaplan et al. 1991; Klee et al. 1991; Nelson et al. 1991; Guydish et al. 1993). Unfortunately, it is not possible on the basis of such surveys to assess whether or not the presence of a needle exchange is associated with the occurrence of HIV or the behaviors associated with HIV, let alone whether any presumed associations are causal in nature. Observations provided by these surveys can be suggestive of causation, but not conclusive.

Suggested Citation:"EVALUATION METHODS." Institute of Medicine and National Research Council. 1994. Proceedings--Workshop on Needle Exchange and Bleach Distribution Programs. Washington, DC: The National Academies Press. doi: 10.17226/4552.
×

Controlled epidemiological studies of the efficacy of exchange programs may consist of studies of populations or of individuals, and each type of study can be conduced as either an observational study or an experimental trial. No study design (epidemiological or otherwise) is without its strengths and potential shortcomings.

Observational Studies of Populations (''Ecologic Studies")

The unit of study in this design is a population or community of injection drug users. For example, the occurrence of HIV in communities with needle exchange programs might be compared with that in communities without needle exchanges. Such studies are relatively inexpensive, and often can be completed quickly when they rely on extant data. Differences observed, however, may not be due to the exposure of interest, but rather to differences in the populations or communities being compared. Because the number of communities under study is usually small, there is limited ability to examine other reasons for any differences in rates that are present in the different populations.

For example, Ljungberg et al. (1991) reported an HIV prevalence of 1% among injection drug users in Skäne, a southern province of Sweden which is served by needle exchange programs in the towns of Lund and Mälmo. By contrast, HIV prevalence among a injection drug users sampled in Copenhagen (a 45 minute ferry boat ride away from Lund) was 15%, among amphetamine injectors in Stockholm it was 4-6%, and among heroin injectors in Stockholm it was 45-60%. Neither Copenhagen or Stockholm are served by needle exchange programs. As the investigators acknowledged, there could well be differences other than the presence or absence of needle exchange programs which could account for the observed variation in HIV prevalence. Copenhagen is a large and cosmopolitan city, while Lund is a relatively small rural university town. Within Sweden, Stockholm has a concentration of heroin users (Käll 1992), many among whom are HIV-positive immigrants from high incidence areas abroad (Kerstin Käll, Karolinska Institute, personal communication 1993).

The role of needle exchange in causing a reduction in HIV occurrence would be supported by large differences in HIV associated with needle exchange; demonstrated comparability between the exchange and non-exchange populations with respect to other predictors of HIV; and samples of communities large enough to enable statistical stability and the assessment of potential confounding.

For example, it would be difficult to interpret a comparison of HIV prevalence among injection drug users in Salt Lake City (which does not have a needle exchange program) with a population of injectors in New York City (which does have a needle exchange) because the two communities differ with respect to the timing of the epidemic and many demographic, sociologic, and cultural factors which may be predictors of HIV occurrence. However, comparisons of injection drug users among comparable New York City and New Jersey communities with and without exchanges, could possibly provide more valid evidence for a possible relationship of needle exchange programs to HIV transmission.

Suggested Citation:"EVALUATION METHODS." Institute of Medicine and National Research Council. 1994. Proceedings--Workshop on Needle Exchange and Bleach Distribution Programs. Washington, DC: The National Academies Press. doi: 10.17226/4552.
×
Experimental Trials of Populations (Community Intervention Trials)

Community intervention trials involve choosing a large number of communities, and then randomly or systematically assigning needle exchange programs to some number of the communities. No such trial has been attempted. If the number of communities were sufficiently large, random assignment of communities to needle exchange could achieve comparability with respect to the predictors of HIV. However, if a small number of communities are selected, the randomization process may not achieve such comparability. With an intervention as controversial as needle exchange, random assignment of communities may not be possible, and if it were, "contamination" of the comparison communities might be hard to prevent.

Observational Studies of Individuals

In these studies, the unit of study is the individual, and the sample size is the number of individuals who participated. There are three types of observational studies of individuals; cross-sectional, case-control, and cohort studies.

Cross-Sectional Studies

Many studies of needle exchange are essentially cross-sectional in nature. In such studies, the exposure (in this case needle exchange use) and the outcomes (e.g., HIV infection) are measured at the same point in time. Cross-sectional studies are usually less expensive than other observational studies, and can often be completed in relatively short time periods. The chief limitation of cross-sectional studies of needle exchange is the inability to determine which came first; the exposure (to needle exchange) or the outcome (for instance, HIV infection or behavior change).

For example, in prevalence surveys among injection drug users entering drug treatment programs, we found that HIV prevalence was higher among those who patronized the needle exchange than among those who did not (Harris et al., n.d.). From these data alone, it is not clear whether knowledge of HIV positivity led people to use the exchange, or exchange use led to acquisition of HIV infection.

Case-Control Studies

In a case-control study, the investigator compares a group with a particular outcome (for example, newly acquired HIV infections) with a comparable group of controls without the outcome (those free of HIV infection), to determine whether or not the groups differ with respect to exposure to needle exchange in the past (prior to acquiring HIV infection, or, in the case of controls, prior to a concurrent time period). Case-

Suggested Citation:"EVALUATION METHODS." Institute of Medicine and National Research Council. 1994. Proceedings--Workshop on Needle Exchange and Bleach Distribution Programs. Washington, DC: The National Academies Press. doi: 10.17226/4552.
×

control studies, like cross-sectional studies, are less expensive and more quickly completed than are cohort studies. However, if the study requires asking subjects to recall past events, their inaccuracy in doing so may lead to a biased result. Also, in a case-control study it care must be taken to establish that the outcome truly followed exposure to needle exchange, and did not precede it.

For example, we conducted a case-control analysis in which we interviewed 2,500 drug injectors in Seattle, systematically recruited in a variety of settings. We defined as cases those who in the 12 months before interview either increased or maintained injecting with non-sterile needles, and we defined as controls those who either decreased or continued not to do so. We then compared the proportion of cases and of controls who had used a needle exchange prior to the 12 month period for which outcomes were measured, to clearly establish a temporal sequence between exposure to the exchange and injection behavior outcomes.

Cohort Studies

In a cohort study, the investigator classifies individuals with respect to exposure and follows them over time to assess subsequent outcomes. Cohort studies generally afford a greater opportunity to evaluate the temporal sequence between needle exchange use and the outcomes of interest than do other observational designs. Aside from the expense and the length of time it takes to complete them, a major consideration in conducting cohort studies, particularly of cohort studies of injection drug users, is the importance of maximizing follow-up.

A cohort study was conducted by Hartgers et al. (1989) in which they interviewed 72 injection drug users in Amsterdam who usually obtained their needles from an exchange, and 73 injection drug users who never or only irregularly used the exchange. A second interview was administered one year later.

If those who are successfully followed in a cohort study differ from those not followed, study findings may be distorted because observed differences in outcome between the exposed and the non-exposed might be due to factors connected with follow-up instead of, or in addition to, differences in the exposures of interest. The authors of the Amsterdam cohort study note that only 49% of exchangers, and even fewer non-exchangers (34%), returned for the follow-up interview, and that those successfully followed differed in important ways from those not followed. The differences between those followed and those not followed could conceivably account for the differences in outcome that they observed between exchangers and non-exchangers.

Experimental Trials of Individuals

In a randomized trial of the impact of a needle-exchange program, participants would be randomly assigned to either needle exchange or to no intervention (or to a different intervention). Subjects would be followed over time, just as in a cohort study,

Suggested Citation:"EVALUATION METHODS." Institute of Medicine and National Research Council. 1994. Proceedings--Workshop on Needle Exchange and Bleach Distribution Programs. Washington, DC: The National Academies Press. doi: 10.17226/4552.
×

to determine subsequent outcomes. The advantage to such studies is that they avoid the distortion in results that might occur if those who self-select to attend a needle exchange program differ from those who don't with respect to risk behaviors. In fact, however, it may not be ethically, politically, or logistically feasible to assign some individuals to needle exchange and prevent others from attending, and as far as we know, no experimental trials have been carried out.

ISSUES OF METHODOLOGICAL BIAS

In epidemiologic studies, bias is present when the observed association between exposure and outcome does not accurately portray the (theoretical) true association. Although the lines between them are occasionally hard to draw, it is useful to consider three general categories of bias: confounding; selection; and information bias.

Confounding

Confounding is the distortion of the true association (or lack thereof) between an exposure and an outcome that is caused by the presence of another exposure or characteristic which also leads to the outcome of interest. In order for a factor to confound, it must be associated both with the outcome and with the exposure.

Sexual behavior (particularly male homosexual behavior) is an example of a potentially confounding factor which should be considered in studies of the impact of needle exchange on the risk of HIV. If men who have sex with men are either more or less likely than others to patronize needle exchange programs (that is, if male homosexuality is associated with the exposure), studies which fail to deal with this fact in design, sampling, or analysis could be confounded and so biased in their results. Nonetheless, few studies of needle exchange have controlled for such behavior.

An example of a study which did is that of Nelson et al. (1991), in which the investigators considered in the analysis the potential confounding effects of receptive anal intercourse in their study of the relationship of diabetes (who have access to sterile needles) to HIV prevalence among injection drug users.

There are a number of ways to deal with the problem of confounding.

  • In design, one can match or stratify on potential confounders.

  • In sampling, one can recruit index and comparison groups from the same underlying target populations in a representative (probabilistic) fashion.

  • In analysis, one can conduct stratified, logistic regression, or other multivariate analyses which incorporate information on potential confounding factors and thus, permit assessment of, and adjustment for, bias resulting from confounding. Such

Suggested Citation:"EVALUATION METHODS." Institute of Medicine and National Research Council. 1994. Proceedings--Workshop on Needle Exchange and Bleach Distribution Programs. Washington, DC: The National Academies Press. doi: 10.17226/4552.
×

analyses are possible, of course, only if relevant data on potential confounders have been collected.

Recently, investigators have begun to consider the effect of potential confounding factors in the evaluation of needle exchange programs.

For example, van Ameijden et al. (1992) investigated sex, date of intake visit, borrowing, intake site, and injection frequency as potential confounding factors and found it necessary to adjust for the latter variable when examining the effect of needle exchange.

Selection Bias

Rothman (1986; after Greenland 1977) states that selection bias occurs when "the relation between exposure and disease is different for those who participate and those who would be theoretically eligible for the study but do not participate".

We assessed HIV prevalence among STD clinic patients sampled in three ways: those who self-selected by requesting HIV counseling and testing; those who agreed to participate in a cross-sectional HIV prevalence survey offered to a systematic sample of patients; and the total STD clinic patient roster (through a "blinded" survey of leftover blood samples from all clients). As can been seen in Figure 1, which compares the findings from the three samples among gay and bisexual men, we found that the more the sample reflected self-selection, the lower the prevalence of HIV.

HIV Prevalence by Sampling Strategy Gay/Bisexual Male IDU's STD Clinic 1990-91

It is common in longitudinal studies that those who return for follow-up differ from those who don't. McDougall et al. (1991) followed gay and bisexual men over time, to

Suggested Citation:"EVALUATION METHODS." Institute of Medicine and National Research Council. 1994. Proceedings--Workshop on Needle Exchange and Bleach Distribution Programs. Washington, DC: The National Academies Press. doi: 10.17226/4552.
×

measure HIV seroconversion. They divided the participants into high-risk, medium-risk, and low-risk groups based on behavior reported at baseline. By its end, only 50% of the high risk group remained in the study, compared to 70% of the low-risk group (see Figure 2). The investigators observed a declining HIV incidence among study participants, but note that the greater loss to follow-up of participants at greater risk may have biased the findings, possibly masking a true increase in HIV incidence.

As with confounding, there are strategies available for minimizing selection bias. One can:

  • Define and represent the target population (or a sub-set of the population) for both index and comparison groups;

  • Compare participants and non-participants on any baseline characteristics for which there are data; and

  • Estimate the likely impact of non-participation on the findings.

As an example, Hart et al. (1989) compared study participants and non-participants on a number of relevant variables, and reported no substantial differences between them (although the study was small, and the statistical power to detect differences may have been limited). Another example of efforts to minimize selection bias can be found in van Ameijden et al. (1993).

Information Bias

Such error occurs when either exposure or outcomes are misclassified (when study information is incorrect). If such error occurs differentially across the index and comparison groups, the result would be either an exaggeration or an underestimation of the effect of needle exchange on the outcomes of interest.

For example, if exchangers were under greater pressure than non-exchangers to display socially desirable risk reduction behavior (Kaplan et al. 1991), and so underreported risky acts, the impact would be to overestimate the effect of needle exchange on risk behavior.

As another example, if exchangers were interviewed by one team of interviewers, and non-exchangers were interviewed by another team (as was done by Donoghoe et al. 1989), it is at least conceivable that team differences in training, style, attitude, or other factors might lead to differential misclassification error and so to bias.

If information error is non-differential across exposure or outcome categories the bias will be towards the null hypotheses (that is, towards the hypothesis that there is no effect of exposure on outcome).

For example, case-control studies often rely upon the ability of cases and controls to recall earlier experiences. If both have equal difficulty doing so, the misclassification

Suggested Citation:"EVALUATION METHODS." Institute of Medicine and National Research Council. 1994. Proceedings--Workshop on Needle Exchange and Bleach Distribution Programs. Washington, DC: The National Academies Press. doi: 10.17226/4552.
×

error would be non-differential, and the resulting bias would be towards the null hypothesis.

An important source of measurement error is imprecision in the questions that are asked of study subjects.

For example, in a case-control analysis of the impact of needle exchange, using extant data from a number of different studies, we were limited to existing questions common to those studies. Such questions tended to be simple and imprecise. Knowing that respondents said "yes" to the question "In the past year, did you use bleach to clean needles before you shot up?" was not very helpful, because that answer didn't reveal whether they cleaned every time, half the time, most of the time, or every time they shot up.

Inadvertent measurement error may occur if, after data collection is complete, the investigators wish to consider a new and different model of the putative relationship between exposure and outcome.

Donoghoe et al. (1989) measured behavior change two to four months after initial exchange use. Measurement of outcomes over such short time frames would allow one to assess immediate effects, but would not allow assessment of a long-term effects.

Apart from striving for precision in measurement, one can attempt validation of self-reported exposure and outcome measures. It is also important to carefully consider models of the relationships between exposures and outcomes, and measure each in a manner appropriate to those models.

STATISTICAL POWER AND MEASURES OF PROBABILITY

Evaluation studies of needle exchange need statistical power sufficient to detect changes in outcomes that may be relatively modest in absolute terms, but are still of interest in terms of slowing the epidemic of HIV. For example, small studies that indicate only small, statistically insignificant differences in the frequency of drug use between persons who do and do not use a needle exchange may, because of their limited power, be missing a true effect of public health importance (Hart et al., 1989).

Other considerations affect power and sample size issues as well. Rare outcomes and rare exposures demand larger sample sizes, as may losses to follow-up and changes in exposure status in longitudinal studies (as when non-exchangers at baseline have become exchangers by follow-up). A larger sample size will also be required if one hopes to examine the possibility of modification of effect across subgroups.

In Seattle, for example, we have found that participation in the needle exchange and the risk of HIV infection both vary by drug of choice. If needle exchange had a different impact among heroin injectors than it did among amphetamine injectors, this modification of effect might be missed if the numbers of subjects in each group were too small, and missing it could result in under-estimation over-estimation of the efficacy of the exchange.

Suggested Citation:"EVALUATION METHODS." Institute of Medicine and National Research Council. 1994. Proceedings--Workshop on Needle Exchange and Bleach Distribution Programs. Washington, DC: The National Academies Press. doi: 10.17226/4552.
×

Measures of probability, especially p-values, have the narrow purpose of eliminating chance as an explanation for observed associations. A better approach to the analysis of association is to use estimates of relative risks (or odds ratios), and confidence intervals around those estimates. These measures provide an estimate of the strength and direction of the association, and the precision of the estimate.

GENERALIZABILITY

As noted above, when assessing a study of needle exchange, we can ask the following questions:

Is there an association between needle exchange exposure and the outcomes of interest?

Is the association likely to be due to chance, due to bias, or to causation?

If a causal interpretation is possible, we can ask ourselves about the extent to which the results might pertain to populations beyond the one studied. The results of a study of an officially-sanctioned needle exchange in a large metropolitan area on the east coast might not have much relevance to an un-sanctioned exchange operating in a small city in the midwest.

FUTURE NEEDS

We need to know just what effects needle exchange programs have, how they have their effects, and among which sub-groups they have their greatest effects. The best epidemiological evidence pertaining to these questions will be afforded by large, statistically powerful, well-designed, observational case-control or cohort studies. While randomized clinical trials might offer even stronger evidence of the impact of needle exchange programs, ethical, political, and logistic considerations weigh heavily against their ever being carried out.

This does not mean that we should wait until such studies are completed until we decide to support needle exchange programs in the face of the HIV epidemic. According to Bradford Hill:

All scientific work is incomplete—whether it be observational or experimental. All scientific work is liable to be upset or modified by advancing knowledge. That does not confer upon us a freedom to ignore the knowledge we already have, or to postpone the action that it appears to demand at a given time.

Suggested Citation:"EVALUATION METHODS." Institute of Medicine and National Research Council. 1994. Proceedings--Workshop on Needle Exchange and Bleach Distribution Programs. Washington, DC: The National Academies Press. doi: 10.17226/4552.
×

ACKNOWLEDGMENTS

The authors would like to thank Gail Hansen and Jan Fields (both of the Seattle King County Department of Public Health) for their assistance, knowledge, and patience.

REFERENCES

van Ameijden ECJ et al., 1992, The Harm Reduction Approach and Risk Factors for Human Immunodeficiency Virus (HIV) Seroconversion in Injecting Drug Users, Amsterdam, American Journal of Epidemiology, 136(2):236-243.

van Ameijden ECJ et al., 1993, AA Major Decline in Risk Behavior Over 6 Years Among IDUs, IXth International Conference on AIDS, Berlin, WS-C21-2.


Brettle RP, 1991, HIV and harm reduction for injection drug users, AIDS, 5:125-136.


Donoghoe MC et al., 1989, Changes in HIV risk behavior in clients of syringe exchange schemes in England and Scotland, AIDS, 3(5):267-272.


Greenland S, 1977, Response and follow-up bias in cohort studies American Journal of Epidemiology 106:184-187.

Guydish J et al., Evaluating needle exchange: are there negative effects? AIDS, 7(6):871-876.


van Haastrecht HJA et al., 1991, The Course of the HIV Epidemic among Intravenous Drug Users in Amsterdam, The Netherlands, American Journal of Public Health, 81(1):59-62.

Harris N et al., n.d. The impact of Seattle's needle exchange program in behaviors associated with HIV transmission. Unpublished manuscript.

Hart GJ et al., 1989, Evaluation of needle exchange in central London: behaviour change and anti-HIV status over one year, AIDS, 3(5):261-265.

Hartgers C et al., 1989, The impact of the needle and syringe-exchange programme in Amsterdam on injecting risk behaviour, AIDS, 3(9):571-576.

Hill AB, 1965, The environment and disease: Association or Causation? Proceedings of the Royal Society of Medicine, 58:295-300.

van den Hoek JAR et al., 1989, Risk Reduction among Intravenous Drug Users in Amsterdam under the Influence of AIDS, American Journal of Public Health 79(10):1355-1357.


Käll K, 1992, Effects of Amphetamine on Sexual Behavior of Male IV Drug Users in Stockholm—A Pilot Study, AIDS Education and Prevention, 4(1):6-17.

Kaplan EH, 1993, Let the Needles Do the Talking! Evaluating the New Haven Needle Exchange, Interfaces, 23.

Kaplan EH, 1991, Evaluating Needle-Exchange Programs via Syringe Tracking and Testing (STT), AIDS & Public Policy J, 6(3):109-115.

Klee H et al., 1991, The sharing of injecting equipment among drug users attending prescribing clinics and those using needle-exchanges, British Journal of Addiction, 86:217-223.

Suggested Citation:"EVALUATION METHODS." Institute of Medicine and National Research Council. 1994. Proceedings--Workshop on Needle Exchange and Bleach Distribution Programs. Washington, DC: The National Academies Press. doi: 10.17226/4552.
×

Ljungberg B et al., 1991, HIV Prevention Among Injecting Drug Users: Three Years of Experience from a Syringe Exchange Program in Sweden, Journal of the Acquired Immune Deficiency Syndromes, 4:890-895.


McDougall L et al., 1991, Bias in AIDS cohort studies and an empiric demonstration of Volunteer Bias in an Investigation of Homosexual Men, VII International Conference on AIDS, Florence, W.C. 95.


Nelson KE, 1991, Human Immunodeficiency Virus Infection in Diabetic Intravenous Drug Users, Journal of the American Medical Association, 266(16):2259-2261.


Office of National Drug Control Policy, 1992, Needle Exchange Programs: Are They Effective? ONDCP Bulletin No. 7.


Popper KR, 1965, The Logic of Scientific Discovery, New York, Harper & Row. Rothman, KJ, 1986, Modern Epidemiology, Little, Boston/Toronto, Brown and Company.


Schlesselman JJ, 1982, Case-Control Studies: Design, Conduct, Analysis, Oxford University Press, New York, Oxford.

Smith A et al., 1991, Evaluation of the needle exchange program in Darwin, Venereology, 4(2):76.

Stimson GV et al., 1989, Syringe exchange schemes: a report and some commentaries, British Journal of Addiction, 84:1283-1290.


U.S. General Accounting Office, 1993, Needle Exchange Programs; Research Suggests Promise as an AIDS Prevention Strategy, Report to the Chairman, Select Committee on Narcotics Abuse and Control, House of Representatives, GAO/HRD-93-60 .


Weiss NS, 1981, Inferring causal relationships; Elaboration of the criterion of "dose-response", American Journal of Epidemiology, 113:487-490.

Suggested Citation:"EVALUATION METHODS." Institute of Medicine and National Research Council. 1994. Proceedings--Workshop on Needle Exchange and Bleach Distribution Programs. Washington, DC: The National Academies Press. doi: 10.17226/4552.
×

AN EVALUATION OF NEEDLE AND SYRINGE EXCHANGE IN SAN FRANCISCO

JOHN K. WATTERS, Urban Health Study, Institute for Health Policy Studies, and Department of Family and Community Medicine, School of Medicine, University of California, San Francisco; MICHELLE J. ESTILO, College of Physicians and Surgeons, Columbia University; GEORGE L. CLARK, Prevention Point Research Group, San Francisco, California; and JENNIFER LORVICK, Urban Health Study, Institute for Health Policy Studies, School of Medicine, University of California, San Francisco

INTRODUCTION

The sharing of contaminated injection paraphernalia is a major route for transmission of human immunodeficiency virus (HIV) in the United States (1,2,3) and is one of the principal means by which HIV infection has spread in Italy, Spain, and Thailand (1,4). In the U.S., one quarter (24%) of the 310,780 AIDS cases diagnosed among adults and adolescents through June, 1993 occurred among heterosexual injection drug users (IDUs). An additional 3.5% (10,800) were adults whose sole risk factor was having a sexual partner who injected drugs. Over half (56.2%) of the pediatric AIDS cases diagnosed in the U.S. through June, 1993, were attributed to HIV transmission from mothers who injected drugs themselves or who engaged in sexual activity with injection drug users (5). It is estimated that there are 1.2 million IDUs in the United States, about 15% of whom are believed to be enrolled in drug treatment on any day (6). Successful prevention of further spread of HIV in this population is crucial to national infectious disease prevention objectives (7). An unknown number of Americans in these risk categories are infected with HIV.

In an effort to reduce the sharing of injection equipment, programs have been established which provide sterile needles and syringes to drug users in exchange for their used equipment. Previous studies have reported that syringe exchange programs have played a significant role in lowering rates of needle-sharing in Amsterdam, (8,9); Sweden (10); Australia (11); the United Kingdom (12,13,14); Tacoma, Washington (15); New Haven, CT (16); and New York City (17,18). Other studies have reported that syringe exchange programs have served as sources of referrals into social services, medical services and drug treatment (16,19). In New Haven, researchers reported that new HIV infections among clients of a legal syringe exchange had been reduced by one-third (17).

Opponents of syringe exchange have claimed that these programs will facilitate and therefore increase illicit drug injection. We conducted a study to determine whether syringe exchange is harmful or beneficial as risk reduction for injection drug users. We evaluated an all-volunteer syringe exchange program in San Francisco, CA known as ''Prevention Point." Estimates of the number of injection drug users in San Francisco, range from 13,000 to 16,000 in a city of approximately 740,000. The daily census in drug

Suggested Citation:"EVALUATION METHODS." Institute of Medicine and National Research Council. 1994. Proceedings--Workshop on Needle Exchange and Bleach Distribution Programs. Washington, DC: The National Academies Press. doi: 10.17226/4552.
×

abuse treatment programs in San Francisco is approximately 1,500 individuals. Waiting lists for publicly supported drug treatment slots exist in virtually all clinics.

Three research questions relevant to the health policy debate regarding syringe exchange were used as evaluation criteria: (1) how readily and to what degree has the syringe exchange been used by IDUs; (2) to what degree has syringe exchange stimulated injection drug use through increased injection frequency and recruitment of new users; and (3) to what degree is the use of syringe exchange predictive of abstinence from syringe sharing?

METHODS

Data Sources and Sampling

Data for the study were derived from two sources: (1) the Urban Health Study, a semi-annual survey of IDUs in San Francisco; and (2) Prevention Point syringe exchange program records. The Urban Health Study is a semi-annual cross-sectional study of IDUs recruited in natural settings in three inner-city communities in San Francisco. During the 1987 through 1989 cross-sections, respondents were also sampled in two 21-day drug detoxification clinics. Communities chosen for study were selected for high densities of IDUs relative to other San Francisco neighborhoods as indicated by review of drug treatment program admissions, drug arrest data, and ethnographic studies. All respondents were screened for visible signs of repeated drug injection. After the purpose of the study was explained, and informed consent obtained, respondents were interviewed using a standard questionnaire dealing with AIDS knowledge; medical, drug use, and sexual histories; and known HIV/AIDS risk behaviors. Interviews were conducted by trained interviewers employed by the Urban Health Study. Respondents were paid for their participation, given pretest and posttest counseling, and given referrals to medical and social services by trained staff. In the present study, we used eleven semi-annual cross-sectional surveys collected as part of the Urban Health Study, between January 1987 and June 1992 (n = 6216). Excluded from further analysis were 572 respondents who reported no current injection drug use. The number of interviews included for each of the cross-sections were as follows: Spring 1987, n = 596; Fall 1987, n = 576; Spring 1988, n = 598; Fall 1988, n = 607; Spring 1989, n = 505; Fall 1989, n = 503; Spring 1990, n = 411; Fall 1990, n = 460; Spring 1991, n = 456; Fall 1991, n = 459; Spring 1992, n = 473.

"Prevention Point" is a volunteer-based syringe exchange program which began operating in November 1988 on a San Francisco street corner, and in a second neighborhood using a "mobile" team. The mobile team used a baby perambulator to deliver necessary supplies to a neighborhood that contained many homeless persons. Additional street corner sites were added in May, 1989; September, 1990; and December, 1991. In September, 1992, the original mobile team was reassigned to two different fixed locations. During the study period, all exchange sites operated during evening hours

Suggested Citation:"EVALUATION METHODS." Institute of Medicine and National Research Council. 1994. Proceedings--Workshop on Needle Exchange and Bleach Distribution Programs. Washington, DC: The National Academies Press. doi: 10.17226/4552.
×

(6:00 to 8:00 pm). While technically illegal, the Prevention Point program has operated without major disruption from police and with the tacit approval of two successive mayoral administrations. Program volunteers provide a strict one-for-one exchange in which a sterile, single-use, 27.5-gauge, 0.5 inch, 1 cc, U-100 insulin syringe is exchanged for each syringe deposited in a Sharps biohazardous waste container by the client. Limitations on the number of syringes program clients were permitted to exchange have changed over the course of the study. Prior to May, 1989 a ten syringes per client visit limit was in effect. Between May 1989 and August 1990, this limit was increased to twenty syringes per client visit. In August, 1990 all limits on the number of syringes that could be exchanged were abandoned. Volunteers also distribute one-ounce bottles of bleach, condoms, cotton, and alcohol wipes, and provide referrals to drug treatment, HIV testing and counseling and other social and medical services upon request. Prevention Point records used for this study extend from program implementation (November, 1988) through mid-1992. Client contacts were recorded on a standard form by program personnel each time an individual presented at a Prevention Point site and exchanged at least one syringe.

Outcome Measures

Utilization of syringe exchange program was assessed using three indicators: (1) the number of client contacts and syringes exchanged as reported in program records of Prevention Point syringe exchange program; (2) frequency of visits to syringe exchange as reported by participants in the Urban Health Study from 1989 to 1992; and (3) sources of syringes reported by Urban Health Study participants interviewed between 1987 and 1992. Negative impacts of the syringe exchange program were examined using three variables included in the Urban Health Study dataset: (1) changes in the self-reported frequency of injection over time (1987-1992); (2) changes in the age distribution of the cross-sections (1987-1992); and (3) proportion of respondents reporting first injection during previous year (19891to 1992). Syringe-sharing was examined by assessing the relationship between reported syringe exchange use in the past year and reported needle-sharing based on self-reported number of needle-sharing partners in the 30 days prior to interview. The accuracy of the term "needle-sharing" has been questioned, since IDUs may use previously-used syringes that are not perceived as "shared" (20). Consequently, participants in the Spring 1992 cross-section (N = 473), were asked if they injected during the past 30 days using syringes that they know had been used by someone else, including a close friend or lover. The Pearson product moment correlation coefficient between this variable and reported "needle-sharing" in the past 30 days was robust (r = 0.83, p < 0.01).

1  

Item added to survey questionnaire in Spring 1989.

Suggested Citation:"EVALUATION METHODS." Institute of Medicine and National Research Council. 1994. Proceedings--Workshop on Needle Exchange and Bleach Distribution Programs. Washington, DC: The National Academies Press. doi: 10.17226/4552.
×

Correlates of needle-sharing were identified using a pool of 752 unduplicated respondents from the most recent full year of data available (Fall 1991/Spring 1992). In instances of multiple interviews, only the first observation was used. Out of 932 interviews, 176 "duplicate" observations and 4 observations with missing data concerning syringe-sharing behaviors were dropped from this analysis. The demographic composition of this cross-section closely approximated the entire sample (data not shown).

Statistical Analysis

One-way analysis of variance with Scheffe's test for multiple comparisons was used to identify differences in the mean number of syringes exchanged and the reported frequency of injection over successive cross-sections. Differences in the proportion of IDUs utilizing the syringe exchange >25 times in the past year, and the proportion of new injectors over-time were assessed using the Mantel-Haenszel χ2 test for trend. Differences in odds ratios between cross-sections were tested using Woolfs method, and summary odds ratios were calculated when appropriate (21). For univariate comparisons, two-tailed χ2 tests or Fisher's exact tests were used to examine the relationship between categorical variables and needle-sharing in the past 30 days; Student's t-tests were used to examine the relationship of continuous variables. Odds ratios and 95% confidence intervals were computed for categorical variables. Multiple linear regression (for continuous outcomes) and logistic regression (for categorical outcomes) was used to control for possible cohort effects in comparisons made over multiple cross-sections. Factors predicting needle-sharing were identified using nonhierarchical logistic regression in a Fall 1991/Spring 1992 sub-sample (n = 752). All possible interactions of main effects were tested. Statistical analysis was performed using the Statistical Package for the Social Sciences, Chicago, Illinois (22).

RESULTS

Table 1 presents selected demographic characteristics of the study population. Analyses of cross-sections were performed over the eleven cross-sections of data incorporated in the study. Changes in grant support forced the elimination of the 21-day drug detoxification clinics from the sampling frame in January, 1990. This resulted in two post-January 1990 changes in sample composition: (1) the proportion of IDUs enrolled in drug treatment programs decreased from 37% to 17%; and (2) the percentage of African Americans increased from 47% to 56%, the percentage of Hispanics decreased from 17% to 14%, and the percentage of Caucasians decreased from 36% to 30%. Among street-recruited IDUs, there was no significant change in proportion of respondents enrolled in drug treatment over the 11 study cross-sections. Changes in sampling frame occurred prior to the 1991-1992 cross-section used in our analysis of needle-sharing in Table 2 and Table 3.

Suggested Citation:"EVALUATION METHODS." Institute of Medicine and National Research Council. 1994. Proceedings--Workshop on Needle Exchange and Bleach Distribution Programs. Washington, DC: The National Academies Press. doi: 10.17226/4552.
×
Utilization of Syringe Exchange Program

Client contacts reported by Prevention Point rose steadily from program implementation in late Fall 1988 through Spring 1992, when client contacts peaked at 16,600 contacts over a six month period. During Spring 1989, 7,821 syringes were exchanged as compared with 343,883 syringes exchanged during Spring 1992. The ratio of syringes exchanged to clients who presented at the exchange sites increased from two syringes per client contact to 21 syringes per client contact between Fall 1988 and Spring 1992. There was a significant increase in the mean number of syringes exchanged for other people on the part of Urban Health Study respondents between Fall 1989 and Spring 1992 (F = 6.603; p < 0.0001). In Fall 1989, 50 respondents (9.9%) reported exchanging syringes for a mean of 4.3 others, while in the Spring 1992 cross section, 72 respondents (15.2%) reported exchanging syringes for a mean of 10.3 others. Between Fall 19892 and Spring 1992, reported utilization of syringe exchange at any time in the past year by respondents in the Urban Health Study increased from 50% to 61%. The proportion of respondents who reported using the syringe exchange at least 25 times in the past year doubled between 1989 and 1992, from 14% to 28% (Mantel Haenszel χ2 test for trend = 40.26; df = 1; p < 0.00001).

We also found major shifts in the principal sources of syringes reported by Urban Health Study respondents between 1987 and 1992. An increase was observed in the proportion of respondents who reported syringe exchange as their usual source. By Fall 1990, syringe exchange had become the most frequently cited source of syringes and remained the major source of syringes throughout the observation period. In Spring 1992, 45% of respondents interviewed reported "usually" obtaining their injection equipment by exchanging at Prevention Point. Thirty-two percent reported that buying syringes on the street was their usual source, while 23% reported using other sources including friends, relatives, diabetics, pharmacies, dealers, shooting galleries, renting or stealing syringes see (see Figure 1).

Frequency of Injection

The median number of reported daily injections in the year prior to interview declined between 1987 and 1992 from a high of 1.9 per day in Fall 1987 to 0.7 in Spring 1992. Median daily injection frequencies in the past year peaked prior to implementation of syringe exchange. This decline in injection frequency over time was significant in analysis of variance over the eleven cross-sections (F = 16.17; p < 0.0001). Scheffe's test for multiple comparisons revealed a significant decline (p < 0.05) between the Spring 1987 to Fall 1988 cross-sections, and between the Fall 1990 and Spring 1992 cross-sections.

2  

Needle-exchange item first added in Fall 1989 survey.

Suggested Citation:"EVALUATION METHODS." Institute of Medicine and National Research Council. 1994. Proceedings--Workshop on Needle Exchange and Bleach Distribution Programs. Washington, DC: The National Academies Press. doi: 10.17226/4552.
×
Recruitment of New and Younger Users into Injection Drug Use

Over the five and one-half year study period, the mean age rose six years, from 35.8 years in Spring 1987 to 41.6 years in Spring 1992. Age was normally distributed, and the mean standard deviation for the eleven cross-sections was 8.3 years. Minimum age did not change significantly between cross-sections. The mean age of the youngest participant across samples was 19; minimum age ranged from 15 to 20. We found a significant progressive decline in the proportion of persons who reported first injecting drugs in the previous year, from 3.0% in Spring 19893 to 1.1% in Spring 1992 (Mantel-Haenszel χ2 test for trend = 9.65; df = 1; p < 0.002).

Syringe Sharing-Univariate Analysis

An overall decline in sharing behavior was observed throughout the observation period with 66.3% reporting sharing in Spring 1987 and 35.5% reporting sharing in Spring 1992. We found no remarkable differences in the proportion of non-users of the exchange who reported sharing needles over the three-year observation period (1989-1992) following program implementation. When all observations from 1989-1992 were considered, IDUs who reported syringe exchange use > 25 times in the past year were less likely to report needle-sharing in the past 30 days than those who used the exchange less frequently or not at all (Mantel-Haenszel summary odds ratio = 0.71; 95% confidence interval = 0.59, 0.87).

The unduplicated Fall 1991/Spring 1992 sample (n = 752) closely approximated the demographic composition of the total sample. Univariate relationships to needle-sharing in the past 30 days from variables in the Urban Health Study data-set may be found in Table 2. A smaller proportion of African-Americans reported needle-sharing in the past 30 days. Homelessness, reported injection of "speedballs" (concurrent injection of heroin and cocaine), heroin injected alone, injected-cocaine, and crack cocaine use in the past month, had significant associations with needle-sharing. Daily injection drug use ( ≥ 30 injections in past 30 days) and a history of drug treatment within the past five years were also associated with needle-sharing. Both older age and needle-exchange as primary syringe source had protective effects. Respondents who reported regular use of bleach within the past six months were less likely to share syringes, as were respondents who reported use of condoms 100% of the time during sexual activity.

3  

Age of first injection was introduced in the Spring 1989 questionnaire.

Suggested Citation:"EVALUATION METHODS." Institute of Medicine and National Research Council. 1994. Proceedings--Workshop on Needle Exchange and Bleach Distribution Programs. Washington, DC: The National Academies Press. doi: 10.17226/4552.
×
Syringe Sharing-Multivariate Analysis

In logistic regression, we found six main effects independently associated with needle-sharing in the past 30 days (see Table 3). Greater frequency of syringe exchange use in the past year was associated with not sharing syringes in the past 30 days. Other protective factors associated with sharing syringes were: increasing age, African American ethnicity, reported condom use 100% of the time during penetrative intercourse (anal, oral, vaginal); and having previously received a HIV antibody test result. Frequency of injection of cocaine in the previous month was a significant predictor of needle-sharing. We found a significant interaction between two continuous variables in the logistic regression model which improved the fit of our model to the data. This interaction adjusts for the difference in the effect of syringe exchange use on sharing behavior relative to years of age. The relationship between needle-sharing, age, and frequency of syringe exchange use is illustrated in Figure 2. Curves depict adjusted odds ratios for needle-sharing for selected age groups. The median age (40 years) for the Fall 1991/Spring 1992 subsample (n = 752) was selected as the referent for calculating odds ratios to illustrate the interaction. Figure 2 shows a decline in the likelihood of needle-sharing among 20 and 30-year olds with increasing frequency of syringe exchange use when adjusted for factors in the model. There was no change in needle-sharing likelihood among 40 and 50 year olds with increasing use of syringe exchange.

DISCUSSION

Our findings confirm that IDUs will participate in syringe and needle exchange programs that can be easily approached and negotiated. In San Francisco, syringe exchange was readily adopted by IDUs, and appears to have quickly replaced the black market as a primary source of injection equipment. The ability of syringe exchange to recapture used and potentially infectious syringes for safe disposal should also not be underestimated. During October, 1992, the Prevention Point syringe exchange program collected and safely incinerated approximately 13,000 used syringes each week. In a recent study, investigators detected HIV-1 antibodies in 7% of a random sample of 83 syringes returned to the San Francisco syringe exchange program (23). By extrapolation, approximately 3,600 syringes contaminated with HIV were removed from the environment during the month of October, 1992 by the syringe exchange program. There was no support for the hypothesis that syringe and needle exchange contributes to drug abuse in our study population. The gradual but statistically significant decline in self-reported frequency of injection over the study period may reflect a historical artifact which mirrors growth in the popularity of "crack" cocaine (which is smoked, not injected), or other unmeasured factors. We found a decreasing level of initiation into drug injection over-time. Guydish et al. (24) noted similar declines in needle-sharing and initiation into injection drug use among persons admitted to drug-abuse treatment programs in San Francisco before and after implementation of the syringe exchange program. Fluctuations in drug use practices are common (25), and initiation into drug

Suggested Citation:"EVALUATION METHODS." Institute of Medicine and National Research Council. 1994. Proceedings--Workshop on Needle Exchange and Bleach Distribution Programs. Washington, DC: The National Academies Press. doi: 10.17226/4552.
×

use (including the practice of injection) have been shown to be influenced by the interplay of a host of familial, social, psychological, cultural, and historical factors (26,27,28,29,30,31). Previous studies reported needle-sharing was lowered among non-exchangers as well as regular exchangers in the Netherlands and the United Kingdom (32,33). However, in our study, we found no significant decline in needle-sharing over-time among respondents who reported no syringe exchange use in the year prior to interview. These differences in study outcomes may stem from different drug use patterns, exchange program structures, and/or social circumstances found among European IDUs and those studied in San Francisco.

We found syringe exchange use to be a strong, independent predictor of not sharing needles and syringes in the recent past when adjusted for age, ethnicity, previous HIV testing and counseling, frequency of injection of cocaine, and consistent use of condoms. Younger IDUs were more likely to report needle-sharing than their older counterparts overall, but were less likely to report needle-sharing with more frequent use of syringe exchange. This finding is reflected in the interaction of age and frequency of syringe exchange use in multivariate analysis. Thus, the San Francisco syringe exchange program appears to have had its greatest benefit among younger users. We speculate these younger persons depend more heavily upon syringe exchange as a source of clean injection equipment than do older IDUs with other established sources for needles and syringes.

Other factors with independent relationships to needle-sharing were race, condom use, and injection of cocaine. It is noteworthy that African American ethnicity was inversely related to syringe sharing. Other studies have found African American IDUs to be at elevated risk for HIV infection relative to IDUs who are members of other ethnic groups in San Francisco (34,35) and elsewhere (36,37). African American ethnicity has been found as an independent risk factor for HIV in multivariate models that include injection frequency (35,38). Possible reasons for this include differential reporting bias and/or social factors related to social networks, sexual activity, and sharing practices that are neither well measured nor understood. IDUs who reported consistent use of condoms "all of the time" were less likely to share syringes. We have previously reported that changes in sexual behavior in response to AIDS prevention messages have trailed gains in needle-cleaning and not sharing (39). It is possible that those who are able to negotiate the use of condoms in the intensely personal and subjective domain of human sexual behavior, are likewise better able to adapt their behavior to avoid needle-sharing. Recent injection of cocaine predicted needle-sharing in multivariate analysis. This may reflect the subordination of health concerns to the drive to inject drugs with whatever equipment is available. Cocaine injectors, who typically inject many times during a "run" of one or several days, may have difficulty obtaining a sufficient supply of clean needles. We speculate that prevention programs, including syringe exchange efforts, will need to reach into the environments where these individuals practice high-risk behavior in order to adequately support the adoption of lower risk alternatives to needle-sharing. We think it especially noteworthy that having received a previous HIV test result (which by California law are voluntary, confidential, and include both pre-test and post-test counseling), was a significant, independent predictor of not-sharing needles. This effect

Suggested Citation:"EVALUATION METHODS." Institute of Medicine and National Research Council. 1994. Proceedings--Workshop on Needle Exchange and Bleach Distribution Programs. Washington, DC: The National Academies Press. doi: 10.17226/4552.
×

was independent of the outcome of the HIV antibody test (positive or negative). Our study is limited in that it identifies correlates of sharing syringes but not causes of reduced sharing. As is the case with virtually all survey research, we used self-reports of respondents recruited into the study from the population of interest; in this case IDUs. Consequently, these data may be subject to problems of recall, intoxication, socially desirable responses, and/or other sources of bias (40). However, high validity of self-reported drug use in a multi-site study using similar methods and instrumentation has been reported (41). The targeted samples used were not true random samples, therefore our findings may not generalize to other populations of IDUs. The short (30-day) timeframe used for some items is a limitation. However, the use of longer periods may amplify problems in accuracy of recall. Despite these limitations, the study contains useful information regarding a significant health technology and related policy issue: namely, the feasibility, and potential risks and benefits of syringe exchange as a method of HIV/AIDS prevention for IDUs.

In multivariate analysis, two inverse correlates of syringe-sharing appeared as promising health interventions. These were syringe exchange and voluntary, confidential HIV testing and counseling. While an independent factor associated with not sharing needles, syringe exchange should not be viewed as a substitute for a comprehensive approach to drug abuse treatment and prevention, nor as an infectious disease prevention nostrum. However, our findings are consistent with other studies that have suggested that increased availability of sterile syringes can play a useful and significant role in helping to attenuate the practice of needle-sharing, and the high rate of infectious disease transmission that accompanies this practice (8,9,10,11,12,13,14,15,16,17,18,42). Our results suggest that syringe exchange programs and voluntary HIV testing and counseling help reduce needle-sharing. Such programs should be continued, expanded to meet existing needs, and implemented in areas where not currently available.

ACKNOWLEDGMENTS

A previous version of this paper was read at the VII International Conference on AIDS, Florence Italy, June, 1991. The authors are especially grateful for the support of the American Foundation for AIDS Research (Grant number 001037-7RG). Additional support for this study was derived from the San Francisco Department of Public Health, Office of AIDS (contract # 83-07069); the U.S. Centers for Disease Control (cooperative agreement # U62-CCU902017); and the National Institute on Drug Abuse (grant nos. RO1 DA04212 and UO1-DA06908). Acknowledged are the contributions of the study participants; and the staff of the following organizations: Prevention Point Needle Exchange Program; Haight-Ashbury Free Clinics, Inc.; San Francisco General Hospital Substance Abuse Services; the Mid City Consortium to Combat AIDS; and the Urban Health Study, School of Medicine, University of California, San Francisco. Acknowledged also are the contributions of Yu-Teh Cheng, M.A., Kyung Hee Choi, Ph.D., M.P.H., Brian Edlin, M.D., T. Stephen Jones, M.D., Walter Hauck, Ph.D., Alex Kral, M.S., Mark Segal, Ph.D., Starley B. Shade, B.A., and David Vlahov, Ph.D.

Suggested Citation:"EVALUATION METHODS." Institute of Medicine and National Research Council. 1994. Proceedings--Workshop on Needle Exchange and Bleach Distribution Programs. Washington, DC: The National Academies Press. doi: 10.17226/4552.
×

REFERENCES

1. Wodak A, Moss A. HIV and injecting drug users: from epidemiology to public health. AIDS 1990;4(suppl):S105-S109.

2. National Commission on AIDS. Report: The twin epidemics of substance abuse and HIV. National Commission on AIDS. Washington, DC; July, 1991.

3. Choopanya K, Vanichsensi S, Plangsringarm K, et al. Risk factors and HIV seropositivity among injecting drug users in Bangkok. AIDS 1991;5(12):1509-1514.

4. Chin J. Epidemiology keynote speech. VII International Conference on AIDS. Florence Italy: 1991.

5. CDC. Report: HIV/AIDS surveillance report, second quarter edition. U.S. Dept. of Health and Human Services, Public Health Service, Centers for Disease Control, National Center for Infectious Diseases, Division of HIV/AIDS. July, 1993.

6. Wiley JA, Samuel MC. The prevalence of HIV infection in the USA. AIDS 1989;3(Supplement 1):71-78.

7. US Department of Health and Human Services. Healthy People 2000: National Health Promotion and Disease Prevention Objectives. Washington, DC: U.S. Government Printing Office (DHHS publication number No. [PHS] 91-50212, 1991.

8. Hartgers C, Buning EC, van Santen GW, et al. The impact of the needle and syringe exchange programme in Amsterdam in injecting risk behavior. AIDS 1989;3:571-576.

9. van den Hoek JA, van Haastrecht HJ, Coutinho RA. Risk reduction among intravenous drug users in Amsterdam under the influence of AIDS. Am J Public Health 1989;10:1355-1357.

10. Ljunberg B, Christensson B, Tuning K, et al. HIV prevention among injecting drug users: three years experience from a syringe exchange program in Sweden. JAIDS 1991;4:890-895.

11. Buzolic A. Report: Needle and syringe availability and AIDS prevention: Modifications of existing legislation. Australia Ministerial Task Force on Drug Strategy. Queensland, Australia; June, 1988.

12. Stimson GV. Syringe exchange programmes for injecting drug users [Editorial Review]. AIDS 1989;3:253-260.

13. Pye M, Kapila M, Buckley G, Cunningham D. A comparative study of local AIDS programs in the United Kingdom. V International Conference on AIDS. Montreal, Canada: 1989.

14. Hart GJ, Carvell ALM, Woodward N, et al. Evaluation of needles exchange in central London: behaviour change and anti-HIV status over one year. AIDS 1989;3:261-265.

15. Purchase D, Hagan H, Des Jarlais DC, Reid T. Historical account of the Tacoma syringe exchange. V International Conference on AIDS. Montreal, Canada: 1989.

16. Kaplan EH, Heimer R. HIV prevalence among intravenous drug users: model-based estimates from New Haven's legal needle exchange. JAIDS 1992;5:163-169.

Suggested Citation:"EVALUATION METHODS." Institute of Medicine and National Research Council. 1994. Proceedings--Workshop on Needle Exchange and Bleach Distribution Programs. Washington, DC: The National Academies Press. doi: 10.17226/4552.
×

17. New York City Department of Health. Report: The pilot needle exchange study in New York City: A bridge to treatment. A report on the first ten months of operation. New York City Department of Public Health. New York; December, 1989.

18. Des Jarlais DC, Hagan H, Purchase D, et al. Safer injection among participants in the first North American syringe exchange program. V International Conference on AIDS. Montreal, Canada: 1989.

19. Carvell AM, Hart GJ. Help-seeking and referrals in a needle exchange: a comprehensive service to injecting drug users. British Journal of Addictions 1990;85:235-240.

20. Grund JP, Kaplan CD, Adriaans NF, et al. The limitations of the concept of needle sharing: the practice of front loading [letter]. AIDS 1990;4:819-821.

21. Schlesselman JJ. Case-Control Studies: Design, Conduct Analysis. New York:Oxford University Press, 1982.

22. SPSS, Inc. SPSS Reference Guide. Chicago: SPSS Inc., 1990.

23. Guydish J, Clark G, Garcia D. Detecting HIV antibodies in needle exchange syringes (letter). AIDS 1992;6:739-740.

24. Guydish J, Bucardo J, Young M, et al. Evaluating needle exchange: are there negative effects? AIDS 1993;7(6):871-876.

25. Kozel NJ, Adams EH. Epidemiology of drug abuse: an overview. Science 1986;21:970-974.

26. Stimson G. Plenary. VII International Conference on AIDS. Barcelona, Spain: 1992.

27. Ellickson PL, Hays RD, Bell RM. Stepping through the drug use sequence: longitudinal scalogram analysis of initiation and regular use. Journal of Abnormal Psychology 1992;101:441-451.

28. Dinwiddie SH, Reich T, Cloniger CR. Patterns of lifetime drug use among intravenous drug users. J Subst Abuse 1992;4:1-11.

29. Dinwiddie SH, Reich T, Cloniger CR. Prediction of intravenous drug use. Comprehensive Psychiatry 1992;33:172-179.

30. Catalano RF, Morrison DM, Wells EA, et al. Ethnic differences in family factors related to early drug initiation. Journal of Studies on Alcoholism 1992;53:208-217.

31. Morrison V. Starting, switching, stopping: users' explanations of illicit drug use. Drug and Alcohol Dependence 1991;27 :213-217.

32. Donoghoe MC, Dolan KA, G. V. S. Changes in injectors' HIV risk behaviour and syringe supply in UK 1987-1990. VII International Conference on AIDS 1991; Book 2(abstract # TH.C.45):66.

33. Hartgers C, van Ameijden EJC, van den Hoek JAR, Coutinho RA. Needle sharing and participation in the Amsterdam syringe exchange program among HIV-seronegative injecting drug users. Public Health Rep 1992;107(6):675-681.

34. Watters JK, Cheng Y, Segal M, et al. Epidemiology and prevention of HIV in intravenous drug users in San Francisco, 1986-1989. VI International Conference on AIDS. San Francisco, CA: 1990.

35. Chaisson RE, Bachetti P, Osmond D, et al. Cocaine use and HIV infection in intravenous drug users in San Francisco. JAMA 1989;261:561-565.

Suggested Citation:"EVALUATION METHODS." Institute of Medicine and National Research Council. 1994. Proceedings--Workshop on Needle Exchange and Bleach Distribution Programs. Washington, DC: The National Academies Press. doi: 10.17226/4552.
×

36. Koblin BA, McCusker J, Lewis BF, Sullivan JL. Ethnic/ethnic differences in HIV-1 seroprevalence and risky behavior among intravenous drug users in a multisite study. American Journal of Epidemiology 1990;132:837-846.

37. D'Aquila RT, Peterson LR, Williams AB, Williams AE. Race/ethnicity as a risk factor for HIV-1 infection among Connecticut intravenous drug users. Journal of Acquired Immunodeficiency Syndromes 1989;2:503-513.

38. Schoenbaum EE, Hartel DH, Selwyn PA, et al. Risk factors for human immunodeficiency virus infection in intravenous drug users. New Eng J Med 1989;321:874-879.

39. Watters JK, Downing M, Case P, et al. AIDS prevention for intravenous drug users in the community: street-based education and risk behavior. Am J Community Psychol 1990;18(4):587-96.

40. Lampinen T, Wiebel WW, Watters JK. Intravenous drug users, HIV testing and counseling [Letter]. JAMA 1989;262:1331.

41. Watters JK, Needle R, Brown BS, et al. The self reporting of cocaine use [letter]. JAMA 1992;268:2374-2375.

42. Nelson KE, Vlahov D, Cohn S, et al. Human immunodeficiency virus infection in diabetic intravenous drug users. JAMA 1991;266:2259-2261.

Suggested Citation:"EVALUATION METHODS." Institute of Medicine and National Research Council. 1994. Proceedings--Workshop on Needle Exchange and Bleach Distribution Programs. Washington, DC: The National Academies Press. doi: 10.17226/4552.
×

Table 1: Selected Demographic Characteristics: IDUs in San Francisco, 1987-1992 (N=5,644)

 

n

%

AGE1

=30

995

17.6

31-40

2753

48.8

41-50

1408

25.0

=51

487

8.6

GENDER1

Male

3893

69.0

Female

1750

31.0

RACE1

African-American

2546

45.1

Caucasian

1941

34.4

Latino

795

14.1

Other

357

6.3

CURRENTLY IN DRUG TREATMENT1

Yes

1529

27.1

No

4078

72.6

Other Answer

9

.2

EVER IN DRUG TREATMENT (past 5 years)1

Yes

3008

53.4

No

2624

46.5

EMPLOYED1

Yes

1179

20.9

No

4401

78.2

Other Answer

50

.9

CONSIDER YOURSELF HOMELESS2

Yes

1445

32.4

No

2987

66.9

Other Answer

30

.7

1 Missing Cases

2 Question added in 1990 (n=4,462)

Suggested Citation:"EVALUATION METHODS." Institute of Medicine and National Research Council. 1994. Proceedings--Workshop on Needle Exchange and Bleach Distribution Programs. Washington, DC: The National Academies Press. doi: 10.17226/4552.
×

Table 2: Selected Characteristics and Reported Needle-Sharing in Past 30 Days-Fall 1991/Spring 1992 (N=752)

Characteristics

#/Share/n

%

Odds Ratio

95% Confidence Interval

RACE

African-American

114/383

(29.8)

1.00

——

Caucasian

91/215

(42.3)

1.73

1.20-2.49

Latino

40/94

(42.6)

1.75

1.07-2.85

Other

22/59

(37.3)

1.40

0.73-2.57

GENDER

Male

181/532

(34.0)

0.80

0.58-1.11

Female

86/220

(39.1)

 

 

AGE

=30

34/73

(46.6)

1.00

——

31-40

129/314

(41.1)

0.80

0.46-1.38

41-50

84/277

(30.3)

0.50

0.29-0.87

=51

20/88

(22.7)

0.34

0.16-0.70

EMPLOYED

Yes

56/149

(37.6)

1.12

0.77-1.62

No

211/603

(35.0)

 

 

CONSIDER YOURSELF HOMELESS        

Yes

121/300

(40.3)

1.42

1.05-1.93

No

141/438

(32.2)

 

 

EVER IN TREATMENT (past 5 years)        

Yes

142/357

(39.8)

1.42

1.05-1.91

No

125/396

(31.8)

 

 

CURRENTLY IN DRUG TREATMENT        

Yes

34/104

(32.7)

0.87

0.55-1.39

No

225/630

(35.7)

 

 

=30 INJECTIONS (past 30 days)        

Yes

141/356

(39.6)

1.41

1.04-1.90

No

126/396

(31.8)

 

 

INJECTED HEROIN (past 30 days)

Yes

208/546

(38.1)

1.53

1.08-2.17

No

59/206

(28.6)

 

 

INJECTED SPEEDBALL (past 30 days)        

Yes

135/333

(40.5)

1.48

1.10-2.00

No

132/419

(31.5)

 

 

INJECTED COCAINE (past 30 days)        

Yes

144/333

(43.2)

1.83

1.36-2.48

No

123/419

(29.4)

 

 

INJECTED AMPHETAMINES (past 30 days)        

Yes

61/155

(39.4)

1.23

0.86-1.77

No

206/597

(34.5)

 

 

SMOKED CRACK COCAINE (past 30 days)        

Yes

162/401

(40.4)

1.59

1.17-2.15

No

105/351

(29.9)

 

 

PRIMARY SOURCE OF SYRINGE = EXCHANGE        

Yes

94/317

(29.7)

0.64

0.47-0.87

No

173/435

(39.8)

 

 

ALWAYS USE BLEACH (past 6 months)

Yes

101/329

(30.7)

0.69

0.51-0.93

No

166/423

(39.2)  

 

 

ALWAYS USE CONDOMS (past year)        

Yes

10/89

(11.2)

0.20

0.10-0.39

No

257/663

(38.8)

 

 

EVER GIVEN PREVIOUS HIV RESULTS        

Yes

172/555

(31.0)

0.48

0.34-0.68

No

95/197

(48.2)

 

 

Suggested Citation:"EVALUATION METHODS." Institute of Medicine and National Research Council. 1994. Proceedings--Workshop on Needle Exchange and Bleach Distribution Programs. Washington, DC: The National Academies Press. doi: 10.17226/4552.
×

Table 3: Logistic Regression Analysis of Needle Sharing in Past 30 Days-Fall 1991/Spring 1992 (N=752)

Variable

Adjusted Odds Ratio

95% Confidence Interval

p=

SYRINGE EXCHANGE USE1 (10 use increment)

0.54

0.35-0.84

.0064

AGE (10 year increment)

0.59

0.45-0.78

.0001

AFRICAN-AMERICAN (vs others)

0.58

0.42-0.81

.0012

ALWAYS USE CONDOMS2

0.17

0.08-0.35

.0001

PREVIOUS HIV TEST RESULT3

0.48

0.34-0.69

.0001

INJECTION COCAINE USE2 (10 use increment)

1.05

1.01-1.08

.0072

INTERACTION4 (age by frequency of syringe exchange use)

.0210

1 past year

2 past 30 days      

3 ever      

4 see Figure 2      

Suggested Citation:"EVALUATION METHODS." Institute of Medicine and National Research Council. 1994. Proceedings--Workshop on Needle Exchange and Bleach Distribution Programs. Washington, DC: The National Academies Press. doi: 10.17226/4552.
×

Figure 1: ''How do you usually obtain syringes?" San Francisco drug injectors, 1987 a-Syringe exchange begins Nov. 1988/10 syringe limit. b-May 1989/20 syringe limit. c-August 1990/syringe limit is abandoned.

Suggested Citation:"EVALUATION METHODS." Institute of Medicine and National Research Council. 1994. Proceedings--Workshop on Needle Exchange and Bleach Distribution Programs. Washington, DC: The National Academies Press. doi: 10.17226/4552.
×

Figure 2: Adjusted odds of needle sharing (30 days) by age and frequency of syringe exchange us previous year

Suggested Citation:"EVALUATION METHODS." Institute of Medicine and National Research Council. 1994. Proceedings--Workshop on Needle Exchange and Bleach Distribution Programs. Washington, DC: The National Academies Press. doi: 10.17226/4552.
×

USING QUALITATIVE METHODS TO EVALUATE NEEDLE EXCHANGE

SHEIGLA MURPHY, Community Health Works, Institute for Scientific Analysis, San Francisco, California

INTRODUCTION

Injection drug use (IDU) continues to be the second most common risk behavior associated with AIDS in this country and serves as a potential bridge of infectivity to other groups including their non-drug-using sexual partners and children (Centers for Disease Control, 1991a; 1991b; Des Jarlais and Friedman, 1988). IDUs are at increased risk for infection with the AIDS virus through the sharing of needles/syringes, as well as other drug injection paraphernalia (Black, et al., 1986; Chaisson et al., 1987b; Chitwood et al., 1990; Friedland et al., 1985; Hart et al., 1989b; Hopkins, 1988; Inciardi, 1990; Inciardi and Page, 1991; Magura et al., 1989; Marmor et al., 1984; 1987; Page et al., 1991; Schoenbaum et al., 1989).

In the absence of an effective treatment or vaccine, efforts to control the spread of the human immunodeficiency virus (HIV) depend on reduction of risk behaviors (Des Jarlais and Friedman, 1987). Public health interventions have taken the form of prevention campaigns employing the media, educational groups or seminars, and street outreach workers. Decreases in needle sharing have been documented (Chaisson, et al., 1987a; Guydish et al., 1990; Rhodes et al., 1990; Selwyn et al., 1987; Sorensen et al., 1989; Stephens et al., 1991). However, it has also been amply demonstrated that an individual's knowledge of high risk behaviors alone is insufficient to ensure discontinuance of risky activities (Davis-Berman and Brown, 1990; Friedman et al., 1992; Inciardi, 1990; Ottomanelli, et al., 1990; Page et al., 1991).

Thus, despite widespread implementation of legal prevention strategies, the epidemic of HIV infection among injection drug users (IDUs) persists unabated. Consequently, illegal and highly controversial needle/syringe exchanges (NSEs) continue operations in cities across the United States. Currently, there is growing interest among policy makers in information concerning these clandestine AIDS interventions. In this paper, I will describe our ethnographic evaluation of San Francisco's NSE, Prevention Point, focusing the discussion on qualitative methods. I will begin with a brief history of Prevention Point and a description of the study design.

PREVENTION POINT

This full-service exchange has been in operation since 1988. As of May, 1993 services are provided on four evenings a week for two hours at ten stationary locations in

Suggested Citation:"EVALUATION METHODS." Institute of Medicine and National Research Council. 1994. Proceedings--Workshop on Needle Exchange and Bleach Distribution Programs. Washington, DC: The National Academies Press. doi: 10.17226/4552.
×

the Hunter's Point, Mission, Tenderloin, Western Addition, Polk Gulch and Civic Center districts of San Francisco. These are multi-ethnic neighborhoods with sizable needle using populations.

Like most other NSEs, Prevention Point participants exchange on a one-for-one basis (used-for-sterile) in order to reduce the number of contaminated syringes in circulation. Staff also provide information on safer sex and drug use, referrals to drug treatment programs and health care agencies, and tangible items such as bleach, alcohol wipes, cottons and condoms. Program providers act as conduits to other social services, such as drug counseling and referrals to drug treatment programs, health care services, and HIV-related services.

Prevention Point is staffed by extremely dedicated volunteers. For example, on the night of the 1989 Loma Prieta earthquake, while most San Francisco residents huddled together eating strange culinary combinations from melting freezers, volunteer needle exchangers braved darkened streets to provide services (Personal Communication, 1992). Prevention Point provides services in stationary locations to multi-cultural participants by dedicated providers.

STUDY DESIGN AND RESEARCH OBJECTIVES

Data collection will consist of both ethnographic field observations, depth interviews and closed-ended questions. Field observations will be conducted at five1 Prevention Point sites. Depth interviews will be conducted with four subsamples: 50 primary exchangers (IDUs who exchange at program sites); 50 secondary exchangers (IDUs who exchange needles/syringes through primary exchangers); 50 non-exchangers (IDUs who do not exchange) and 25 Prevention Point staff. As of September 1993, we have completed field observations at two sites and depth interviews with 14 providers, 11 primary exchangers, 12 secondary exchangers and 14 non-exchangers.

We are conducting this research in order to elicit the following kinds of information: first and foremost, the affect of NSE on the continuation/reduction of needle sharing, as well as sex-related risk behaviors. We are investigating IDUs' perceptions of the factors which influence NSE participation and the barriers to participation. We are gathering information about secondary users of NSE-IDUs who exchange needles through other people and the nature of their relationship (e.g., drug dealer, partner, customer). We are looking at the role of the NSE protocols (e.g., limitations on exchanges, hours of operation, geographic locations) in increasing or decreasing particular types of IDUs' participation and the ways in which various clients/providers experience the intervention. We are examining the role of provision of other services (e.g., referral to drug treatment) in NSE utilization, as well as IDU's involvement with other public health and social services.

1  

When we submitted the proposal to NIDA to conduct this evaluation, Prevention Point only had five sites. We are currently preparing a competing supplemental application so that we can do participant observations and depth interviews at all ten sites.

Suggested Citation:"EVALUATION METHODS." Institute of Medicine and National Research Council. 1994. Proceedings--Workshop on Needle Exchange and Bleach Distribution Programs. Washington, DC: The National Academies Press. doi: 10.17226/4552.
×

Prevention Point, vis-a-vis other American NSEs, is in a unique position. Shortly after our National Institute on Drug Abuse funded project began in May of 1993, San Francisco's Mayor Jordan declared the city to be in a state of medical emergency. This allowed the Mayor and Board of supervisors to circumvent relevant prescription and paraphernalia laws and allocate public health funds to finance Prevention Point. Thus, our research team is in the very fortunate position of being able to study Prevention Point's transformation from a quasi-legal community-based organization into a publicly-funded social service agency.

In sum, our research objectives are to produce a thorough understanding of the ways in which Prevention Point is currently implemented, utilized and experienced. We will focus on how the program is perceived by participants, non-participants and staff. We will explore the factors which serve as barriers to, or facilitate participation in, NSE. We plan to study not only formal activities and anticipated outcomes, but also informal activities and unanticipated outcomes. Finally, we will provide a detailed description of the impact of this intervention on HIV drug and sex-related risk behaviors. In the following, I will describe the theoretical and methodological perspectives which guide this ethnographic evaluation.

THEORETICAL AND METHODOLOGICAL PERSPECTIVE

In order to achieve our research objectives, we are employing qualitative methods. The philosophical and theoretical perspectives which provide the framework for qualitative methods include phenomenology (Bussis, Chittenden and Amarel, 1973; Carini, 1975), symbolic interactionism and naturalistic behaviorism (Denzin, 1978a; 1978b), and ethnomethodology (Garfinkel, 1967).

A unifying theme running through these varied perspectives is an emphasis on the importance of understanding the meanings of human behavior and the socio-cultural context of interactions. In order to understand any social phenomenon, one must understand the dynamic definitions and patterns of interaction of the social actors. The consequences for methodology of this emphasis is ". . .the qualitative study of people in situ is a process of discovery. One must find out what the subjects themselves believe they are doing in their own terms rather than impose a preconceived or outsider's scheme of what they are doing" (Lofland, 1971:4).

In order to understand Prevention Point's program operations as they are experienced by staff, clients and non-clients, we are conducting a qualitative process evaluation (Rossi and Freeman, 1989). Traditionally, evaluation research has been criticized by social scientists because of its lack of theoretical sensitivity. This qualitative evaluation research employs a inductive grounded theory approach in order to generate program theory from holistic data gathered through naturalistic inquiry (Glaser and Strauss, 1967; Strauss and Corbin, 1990).

This discovery of program theory will help prevention experts and decision makers understand how Prevention Point functions, why it functions the way it does, and the ways in which outcomes flow from program activities. Prevention experts can use

Suggested Citation:"EVALUATION METHODS." Institute of Medicine and National Research Council. 1994. Proceedings--Workshop on Needle Exchange and Bleach Distribution Programs. Washington, DC: The National Academies Press. doi: 10.17226/4552.
×

grounded theory to test their own theories of programmatic action, program effects and the relationship between action and effects. Grounded theory evaluation, through its rich and detailed description, takes decision makers into the empirical world as it is experienced by the actors in the setting. Grounded theory evaluation is particularly efficacious in considerations of whether or not to replicate similar programs in other settings (Patton, 1987). It can also be helpful in discovering feasible alternatives to, or modifications of, existing services.

Process evaluations look not only at formal activities and anticipated outcomes, but they also investigate informal patterns and unanticipated consequences in the full context of program implementation and development. Finally, process evaluations usually include the perspectives of people close to the program about how things are going. A variety of perspectives may be sought from people in dissimilar relationships to the program-inside and outside sources. (Patton, 1987:24)

We are studying program implementation by gathering detailed, descriptive information from both clients and staff about what the program is doing in order to answer the following kinds of questions: Who participates in needle exchange? What do Prevention Point clients experience? What services are provided to clients? What other services do they need? What does the staff do? What is it like to exchange needles? What are the differences between the service sites? How does participation affect their drug/sex HIV risk behaviors?

From IDUs who do not participate in NSE (either not at all or indirectly), we are seeking detailed information about the following areas: What do they know about NSE? Why don't they participate in NSE? What are their perceptions of NSE's services, staff and participants? What services do they need? What is their knowledge of and participation in drug/sex related HIV risk behaviors?

Data Collection

As noted previously, data collection consists of ethnographic field observations, depth interviews and closed-ended questions. Field observations will be conducted at 5 Prevention Point sites. At each of the 5 sites, we are conducting field observations for 4 weeks (one evening a week) before respondent recruitment begins and are continuing observations throughout the data collection period. The purpose of field observations is to describe the process of NSE thoroughly and carefully. Information from these observations informs instrument preparation and sample selection. At each site, two or more project staff observes interactions and conducts informal interviews with program participants. Field workers tape record their field notes immediately following each observational period.

By directly experiencing needle exchange, field observers have intimate knowledge of the context of service provision. Trained ethnographic field workers have the

Suggested Citation:"EVALUATION METHODS." Institute of Medicine and National Research Council. 1994. Proceedings--Workshop on Needle Exchange and Bleach Distribution Programs. Washington, DC: The National Academies Press. doi: 10.17226/4552.
×

opportunity to see things program participants (staff and exchangers) are unaware of due to the routine nature of the experience for them. Field observers form their own understandings of NSE without having to rely totally on the selective understandings/perceptions participants recount during interviews. In sum, the results of our analysis of descriptive data from field notes will allow this study's information users to enter the social world of needle exchange (Patton, 1987; Rossi and Freeman, 1989).

Field workers also conduct observations in the places IDUs congregate (copping areas, bars, etc.) in the neighborhoods surrounding Prevention Point sites. Information from these observations also informs instrument preparation and sample selection. Field notes from these observations are tape recorded and analyzed to develop a more thorough understanding of the social worlds of IDUs who do not exchange.

Depth interviews are being conducted with the four subsamples (providers, primary and secondary exchangers and non-exchangers). We have developed interview guides or lists of topics to be explored. Depth interviewing from an interview guide ensures consistent inquiries with each interview, without precluding the possibility of discovery of other relevant issues. The interview guide serves as a basic checklist. The interviewer is then free to ask questions in a conversational style in a sequence that flows from the respondent's perspective of the situation. Interview guides help focus the interaction while allowing unanticipated topic areas to emerge during the interview.

We also utilize closed-ended questionnaires to enable us to collect relevant demographic data, as well as drug/needle use and NSE utilization history. We include measures of current NSE utilization, needle-sharing and high-risk sexual practices. In addition, questions have been designed to gather information regarding: family of origin; current family and marital relations; work and education; drug and alcohol use histories; arrests and criminal activities; and utilization of social services.

Triangulation

Denzin (1978a) identified four forms of triangulation, or ways of strengthening research designs by building checks and balances into data collection and analysis strategies. We have included three forms of triangulation in the design of this study. The combination of field observations and depth interviews is methodological triangulation, or the multiple use of methods to study a single problem. The process of interviewing staff, participants and non-participants is data triangulation, since we are analyzing program perceptions from diverse perspectives. Finally, each Prevention Point site will be observed by at least two project staff allowing for investigator triangulation. We will then be able to compare field notes from different field workers, further strengthening the resulting information (Denzin, 1978a).

Suggested Citation:"EVALUATION METHODS." Institute of Medicine and National Research Council. 1994. Proceedings--Workshop on Needle Exchange and Bleach Distribution Programs. Washington, DC: The National Academies Press. doi: 10.17226/4552.
×

SAMPLING STRATEGIES

We have chosen a purposeful sampling method employing a maximum variation strategy:

The logic of purposeful sampling in qualitative methods is quite different from the logic of probabilistic sampling in statistics. The power of statistical sampling depends on selecting a truly random and representative sample which will permit confident generalization from the sample to a larger population. The power of purposeful sampling lies in selecting information rich cases for study in depth (Patton, 1987:53).

Maximum variation sampling aims at capturing the central themes that cut across a great deal of participant variation. The logic of maximum variation sampling presumes any common patterns emerging from great variation are of value in understanding the core experiences of program participants. We will be including in the sample individuals who have had quite different experiences informed by our field work observations and informal interviews with clients and staff. "By including in the sample individuals the evaluator determines have had quite different experiences, it is possible to describe more thoroughly the variation in the group and to understand variations in experiences." (Patton, 1987:54)

Providers

Interviews will be conducted with twenty-five Prevention Point staff. Field observations will inform provider interviewee selection procedures with the intention of capturing the core provider perceptions that cut across variation among providers.

Primary Exchangers

We will recruit fifty adult primary exchangers, ten from each of five sites. The ten interviewees selected will represent the range of variation of NSE participants. Selected exchangers will be asked to participate in this voluntary evaluation of NSE. Ethnographers will record basic observable information (e.g., gender, race) on all those who refuse to participate, as well as recruitees, in order to gather some systematic information about individuals who refuse to participate or who do not show up for appointments.

Suggested Citation:"EVALUATION METHODS." Institute of Medicine and National Research Council. 1994. Proceedings--Workshop on Needle Exchange and Bleach Distribution Programs. Washington, DC: The National Academies Press. doi: 10.17226/4552.
×
Secondary Exchangers

We will locate fifty adult secondary users of needle exchange by asking the primary exchangers we interview to refer to the project the individuals for whom they have exchanged. It is difficult to estimate the refusal/no show rate for this sampling procedure. However, if our primary exchangers do not refer enough secondary exchangers, we will recruit other primary exchangers we have not interviewed and ask them to refer respondents until we have 10 interviews from each site or 50 secondary exchangers.

Non-Exchangers

A non-exchanger is an adult IDU who has not been a participant in needle exchange (primarily or secondarily) for three months. They must also have injected at least five times during the past three months. We are including those IDUs who may have stopped exchanging in order to discover the barriers for IDUs who have actually experienced NSE, as well as those who have not. We are going to interview infrequent/new injectors, as well as, frequent/experienced injectors.

Non-exchangers are located via chain referral and field work. Field observations in neighborhoods surrounding Prevention Point sites is being conducted in order to locate non-exchangers for our study. Interviewees are also located by using the snowball sampling or chain referral method (Biernacki and Waldorf, 1981; Watters and Biernacki, 1989). The chain referral method works as follows: one respondent refers us to his/her friends who meet study criteria for the non-exchanger subsample. The referred respondent then refers us to others thus forming chains of respondents. We have initiated chains with interviewees from our current and past drug projects. We select among the non-exchangers located utilizing the maximum variation sampling strategy.

Members of both genders and members of ethnic groups represented at site will be included in each of the four subsamples interviewed from each site.

THE INTERVIEW

The interview process begins with the initial contact with project staff or interviewee/locators. They are briefly informed of confidentiality procedures and an interview appointment will be scheduled as soon as possible. The optimal situation is to interview them on the spot. The time of day, length of interview and interview topics explored often makes scheduling appointments to conduct interviews at a later time, either in their homes or at our field offices, more practical.

At the time of interview, the interviewer begins by acquainting the respondent with the nature of the study and the human subjects protection. After the interviewee has read and signed the consent form, the interview proceeds with the tape-recorded depth interview portion, followed by the closed-ended questions. The process takes

Suggested Citation:"EVALUATION METHODS." Institute of Medicine and National Research Council. 1994. Proceedings--Workshop on Needle Exchange and Bleach Distribution Programs. Washington, DC: The National Academies Press. doi: 10.17226/4552.
×

approximately two to three hours, therefore, we provide a forty dollar honorarium. Likewise, locators require an adequate remuneration (twenty dollars) for their time spent doing the important work of recruiting respondents. Additionally, the interviewee is asked (with the right of refusal) to provide follow-up tracking information so that s/he can be re-contacted for follow-up.

Immediately following field observations, field observers will tape record key descriptive and narrative material. The depth interviews will be recorded on cassette tapes and then coded and archived by project staff. Also following the interview, the interviewer writes up his/her impressions in a one page summary statement.

QUALITATIVE DATA ANALYSIS

We use the grounded theory method in the analysis of the qualitative data (Charmaz, 1983; Glaser and Strauss, 1967; Strauss and Corbin, 1990). This method's premise is data should be collected and analyzed simultaneously to allow the basic social, social psychological and structural processes inherent in a phenomenon to emerge naturally. We employ this method in an effort to both discover new theory or reconstruct existing theory where it is applicable (Burawoy, et al., 1991). The flexibility of this methodology is perhaps most useful because it allows the researcher to correct mistaken hunches or hypotheses prior to completion of the study. As Borman, et al. (1986:34) note:

reports of competently done qualitative studies focus upon the flexible, evolutionary and recursive nature of the investigation; the emphasis of the paradigm is upon remaining sensitive to the data and to input from the field. When initial questions of procedures appear to clash with incoming information, the paradigm permits researchers to abandon unworkable lines of inquiry and reformulate new ones that have a better fit. The resulting nested working hypotheses help guide a course of inquiry that leads toward results that closely adhere to the phenomena and have great authenticity. Rather than simply being an ill-thought-through ad hoc operation, the looseness that characterizes qualitative research is one of its defining features and greatest strengths. It permits the researcher to correct mistakes.

While collecting data, the researcher continually makes theoretical, methodological and observational notes which guide him/her through the data-gathering stage naturally into analysis (Schatzman and Strauss, 1973; Strauss and Corbin, 1990). The investigator begins by coding the data immediately after collection. It is imperative as little time as possible elapses between collection and initial analysis. In the coding, the analyst notes patterns which seem salient due to their recurring nature. Either the interviewee has mentioned a given problem/circumstance over and over again during the session, or the researcher has noticed this problem/circumstance is referred to by many interviewees. Through coding the data for salient dimensions, constellations of basic social, social-psychological and structural processes are discovered.

Suggested Citation:"EVALUATION METHODS." Institute of Medicine and National Research Council. 1994. Proceedings--Workshop on Needle Exchange and Bleach Distribution Programs. Washington, DC: The National Academies Press. doi: 10.17226/4552.
×

While completing open coding, and throughout the data collection process, we write theoretical and methodological memos. Through memoing the data, we then begin to conceptually connect the diverse codes. In addition, this process helps to clarify holes in the analysis, which provides new directions for data collection. Saturation is reached when continuing redundancy is found in basic social processes and their properties among a number of interviewees.

This will be done in an attempt to deal with what Guba (1978) calls the problems of ''convergence and divergence." That is, the discovery of how various pieces of information fit together, or what Strauss and Corbin (1990) called the paradigmatic model. The analyst seeks recurring regularities, between observational and interview data, or patterns that can then be sorted into categories.

Categories should then be judged by two criteria: "internal homogeneity" and "external heterogeneity." The first criterion concerns the extent to which the data that belong in a certain category hold together or dovetail in a meaningful way.... The second criterion concerns the extent to which differences among categories are bold and clear. The existence of a large number of unassignable or overlapping data items is good evidence of some basic fault in the category system (Guba, 1978:53).

The process evaluator then goes back and forth between the data and the emerging paradigm to verify the meaningfulness of the categories and modify the paradigmatic model as necessary.

Along with the tools of the grounded theory method—coding, memoing, and paradigm building (Glaser and Strauss, 1967; Strauss and Corbin, 1990)—we utilize other analytic procedures, in particular domain analysis (Spradley, 1979).

Domain Analysis

Since the focus of this study is the investigation of a new culture, with its own specialized and technical jargon, the analytic tool of domain analysis is extremely useful (Olesen and Whitaker, 1968; Spradley, 1979). Briefly, domain analysis attempts to discover most of a culture's principles for organizing symbols into domains. Spradley (1979:107) describes the importance of the semantic relationship to this discovery process:

A more efficient procedure in identifying domains makes use of the semantic relationship as a starting point. From a growing body of research, it appears that the number of semantic relationships in any culture is quite small, perhaps less than two dozen. In addition, certain semantic relationships appear to be universal. These remarkable facts make semantic relationships an extremely useful tool in ethnographic analysis. Using these relational concepts, the ethnographer can discover most of a culture's principles for organizing symbols into domains.

Suggested Citation:"EVALUATION METHODS." Institute of Medicine and National Research Council. 1994. Proceedings--Workshop on Needle Exchange and Bleach Distribution Programs. Washington, DC: The National Academies Press. doi: 10.17226/4552.
×

Conducting domain analyses on the interview/field note data enriches the resulting findings and acts as a sort of validation process for the other analytic procedures. For example, conducting a domain analyses of the semantic term "needle sharing" from the provider, exchanger and non-exchanger interview/observational data should prove very revealing and significant in increasing our knowledge of the cultural meaning of this high risk behavior.

Coding

The first code list will be derived directly from the interviews and will consist of subject areas which, by virtue of the time the respondent spent discussing them and/or their recurrent nature, seem important. We will also begin with an initial set of codes taken roughly from the interview guide, since we generated the topic areas from our research questions about issues relevant to NSE. For example, from our previous research, we anticipate that interviewees will recall their first needle use experiences as turning points in their drug using careers. Thus, as indicated in the qualitative interview guide, we include specific questions about this subject. In this case, we might title a code "initial injection" and designate the appropriate lines of text. The codes will be revised as the data are analyzed. Codes can be modified, collapsed, expanded or dropped as new codes are added to the code list. If "initial injection" does not turn out to be a salient issue after initial coding procedures, the text can be re-coded. On the other hand, if we encounter new subject areas which interviewees discuss (despite their absence from the formal interview guide), we will code them and then incorporate them into the interview guide.

Memoing

The interview will be memoed after coding, at which time the analyst makes theoretical and methodological notes about the data. Basically, the question "What is going on here?" is addressed in a memo. The memos vary in length and often contain direct quotes from the interview that might ultimately be used as data in published material. The memoing is also done on the computer, and the memos are filed according to the code to which they correspond. For example, a passage in the interview in which a provider was discussing her opinions regarding a particular program protocol might be coded "Protocol". We might then make a memo about this activity, possibly including a direct quote from the provider. The memo would then be entered in the ''Program Protocol" file, and other such memos would be entered into the same file. Memo-making is the hallmark of the method to be used in this study. While coding the data for prominent dimensions and certainly afterward, the analyst makes theoretical memos to be used in the analysis (Glaser, 1978).

Suggested Citation:"EVALUATION METHODS." Institute of Medicine and National Research Council. 1994. Proceedings--Workshop on Needle Exchange and Bleach Distribution Programs. Washington, DC: The National Academies Press. doi: 10.17226/4552.
×
Field Notes

Field notes are incorporated into the analysis of the data in much the same manner as interview data. The field notes are coded and then entered into Folio Views. They are then entered in the appropriate file and are ready for memo-making. These memos are considered data and are treated as such, since interview and field data are complementary. A field worker, in fact, may inadvertently collect interview data while doing field observations, or when interviewing, the interviewer may observe occurrences which might more technically be defined as field observations. In sum, interview and field data, though separate, operate together in the analysis of data.

Data Organization

The data (field notes and memos) is organized according to the code list. Next, coded materials from the interview transcript, the memos and the field notes are entered into code files. A possible category of data, for example, might be "Barriers to NSE." Based on other studies using a similar methodology, we might start with thirty codes and perhaps ultimately collapse them to fifteen.

After approximately five interviews have been transcribed, coded and memoed, the research team will read all the memos and, based upon what they contain, scrutinize the interview guide, possibly emphasizing new areas of investigation. In this way, we probe in areas that are emerging as important and put less emphasis on areas which did not elicit much response and, therefore, did not seem important to the phenomena.

VALIDITY AND RELIABILITY

In exploratory research that is primarily descriptive in nature, validity problems center around these questions: 1) Did the respondent tell the truth? 2) Did the investigators represent the social world, situations and perspectives of the subjects as they see it?

It has been our experience with over seven hundred interviews that, given the assurance of confidentiality, illicit drug users can be relied on to give valid information. Numerous other studies have shown that illicit drug users are usually truthful when interviewed (Ball, 1967; Bonito et al., 1976; Maddux and Desmond, 1975; Nurco, 1975; Stephens, 1972). Nonetheless, we have built into our research some strategies for insuring truthfulness. Many of the respondents are known either directly or indirectly by the interviewers and other respondents. Thus, shared knowledge and mutual experiences make successful deception more difficult. Additionally, asking the same question, worded slightly differently at another point during the interview, helps to expose any inconsistencies in respondents' answers. This is a long and detailed interview, and even a dedicated liar would have difficulty keeping a false story straight.

Suggested Citation:"EVALUATION METHODS." Institute of Medicine and National Research Council. 1994. Proceedings--Workshop on Needle Exchange and Bleach Distribution Programs. Washington, DC: The National Academies Press. doi: 10.17226/4552.
×

Strategies for ensuring the validity and adequacy of the data include data source triangulation, researcher and respondent validation and the search for negative cases. Since we conduct field observations and depth interviews, this allows us to employ data triangulation. Another method qualitative researchers use is to seek respondent validation. In previous studies, we have used this method by organizing group sessions with three or more participants as a validating tool. Interviewees become more sociological and abstract in these group sessions and often see themselves as responsible for making sociological sense out of the data in question.

After we develop an analytic framework we are at least provisionally satisfied with, we will informally (while doing fieldwork) interview respondents to ascertain whether our analysis rings true for them. We will conduct these informal interviews with members of each study group with representatives from each site. This validation process will be ongoing from the beginning of data collection and analysis.

Working with a mixed gender multi-ethnic research team will provide the opportunity for researcher triangulation. If the same themes and categories appear in each of our interviews, it will serve as a researcher reactivity check (Hammersley and Atkinson, 1983). Additionally, all the field workers/interviewers will read both memos and drafts of various chapters to verify conceptual relations (Strauss and Corbin, 1990).

In all project publications, we will provide detailed descriptions and in-depth quotations in order to allow readers to understand the program fully and the perspectives of the people represented in this study. This description will be balanced by analysis and interpretation. Verification and validation information will be added wherever relevant. We will include throughout the text explanations concerning the extent of triangulation, validity checks, and supporting evidence for our theoretical framework. This will permit readers to critically evaluate the fit between the data and the analysis.

BIBLIOGRAPHY

Ball, J. (1967) 'The Reliability and Validity of Interview Data Obtained from 59 Narcotics Addicts," American Journal of Sociology 72:650-654.

Biernacki, P. and Waldorf, D. (1981) "Snowball Sampling: Problems, Techniques and Chain-Referral Sampling," Sociological Methods and Research. 10(2):141-163.

Black, J., Dolan, M., DeFord, H., et al. (1986) "Sharing of Needles among Users of Intravenous Drugs," The New England Journal of Medicine, 314:446-447.

Bonito, A., Nurco, D. and Shaffer, J. (1976) "The Verdicality of Addicts' Self-Reports in Social Research," The International Journal of the Addictions 11(5):719-724.

Borman, K., LeCompte, M., Goetz, J. (1986) "Ethnographic and Qualitative Research Design and Why it Doesn't Work," American Behavioral Scientist. 30(1):42-57, September/October.

Burawoy, M., Burton, A., Ferguson, A., Fox, K., Gamson, J., Gartrell, N., Hurst, L., Kurzman, C., Salziner, L., Schiffman, J. and Ui, S. (1991) Ethnography Unbound: Power and Resistance in the Modem Metropolis. Berkeley and Los Angeles, CA:University of California Press.

Suggested Citation:"EVALUATION METHODS." Institute of Medicine and National Research Council. 1994. Proceedings--Workshop on Needle Exchange and Bleach Distribution Programs. Washington, DC: The National Academies Press. doi: 10.17226/4552.
×

Bussis, A., Chittendon, E., and Amarel, M. (1973) Methodology in Educational Evaluation and Research. Princeton, N.J.: Educational Testing Service.

Carini, P. (1975) Observation and Description: An Alternative Methodology for the Investigation of Human Phenomena. Grand Forks: University of North Dakota Press.

Centers for Disease Control. (1991a) AIDS/HIV Surveillance, National Commission on AIDS Report, July.

Centers for Disease Control. (1991b) The HIV/AIDS Epidemic: The First 10 Years, Morbidity and Mortality Weekly Report, 40:357-363, 369, June 7.

Chaisson, R., Osmond, D., Moss, A., et al. (1987a) "HIV, Bleach, and Needle Sharing," Lancet I, 8547:1430.

Chaisson, R., Moss, A., Onishi, R., Osmond, D. and Carlson, J. (1987b) "Human Immunodeficiency Virus Infection in Heterosexual Intravenous Drug Users in San Francisco," American Journal of Public Health, 77(2):169-172, February.

Charmaz, K. (1983) "The Grounded Theory Method: An Explication and Interpretation." In Contemporary Field Research (Ed.) R. Emerson, 109-126. Boston: Little-Brown.

Chitwood, D., McCoy, C. and Comerford, M. (1990) "Risk Behavior of Intravenous Cocaine Users: Implications of Intervention." In National Institute on Drug Abuse Research Monograph 93 (AIDS and Intravenous Drug Use: Future Directions for Community-Based Prevention Research), C. Leukefeld, R. Battjes and Z. Amsel (Eds.), National Institute on Drug Abuse Office of Science, Rockville, Maryland.


Davis-Berman, J. and Brown D. (1990) "AIDS Knowledge and Risky Behavior by Incarcerated Females: IV and Non-IV Drug Users," Social Science Review, 75(1):816, October.

Denzin, N. (1978a) "The Logic of Naturalistic Inquiry." In N. Denzin (Ed.) Sociological Methods: A Sourcebook, New York: McGraw-Hill.

Denzin, N. (1978b) The Research Act. New York:McGraw-Hill.

Des Jarlais, D. and Friedman, S. (1988) "The Psychology of Preventing AIDS among Intravenous Drug Users: A Social Learning Conceptualization," American Psychology, 48:865-870.

Des Jarlais, D. and Friedman, S. (1987) "Target Groups for Preventing AIDS Among Intravenous Drug Users," Journal of Applied Social Psychology, 17(3):251-268.


Friedland, G., Harris, C., Butkus-Small, C., et al. (1985) "Intravenous Drug Abusers and the Acquired Immunodeficiency Syndrome (AIDS)," Archives of Internal Medicine, 145:1413-1417.

Friedman, S., Neaigus, A., Des Jarlais, D., Sotheran, J., Woods, J., Sufian, M., Stepherson, B. and Sterk, C. (1992) "Social Intervention Against AIDS among Injecting Drug Users," British Journal of Addiction, 87:393-404.


Garfinkel, H. (1967) Studies in Ethnomethodology. Englewood Cliffs, N.J.: Prentice-Hall.

Glaser, B. (1978) Theoretical Sensitivity. Mill Valley, CA:The Sociology Press .

Glaser, B. and Strauss, A. (1967) The Discovery of Grounded Theory. Chicago:Aldine.

Suggested Citation:"EVALUATION METHODS." Institute of Medicine and National Research Council. 1994. Proceedings--Workshop on Needle Exchange and Bleach Distribution Programs. Washington, DC: The National Academies Press. doi: 10.17226/4552.
×

Guba, E. (1978) Toward a Methodology of Naturalistic Inquiry in Educational Evaluation. CSE Monograph Series in Evaluation No. 8. Los Angeles: Center for the Study of Evaluation.

Guydish, J., Abramowitz, A., Woods, W., Black, D. and Sorensen, J. (1990) "Changes in Needle Sharing Behavior Among Intravenous Drug Users: San Francisco, 1986-88," American Journal of Public Health, 80(8):995-996.

Hammersley M. and Atkinson, P. (1983) Ethnography Principles and Practice. London: Tavistock Publications.

Hart, G., Sonnex, C., Petherick, A., et al. (1989b) "Risk Behaviors for HIV Infection among Injecting Drug Users Attending a Drug Dependency Clinic," The British Medical Journal, 298:1081-1083.

Hopkins, W. (1988) "Needle Sharing and Street Behavior in Response to AIDS In New York City." In National Institute on Drug Abuse Research Monograph 80 (Needle Sharing among Intravenous Drug Abusers: National and International Perspectives), R. Battjes and R. Pickens (Eds.), National Institute on Drug Abuse Office of Science, Rockville, Maryland.


Inciardi, J. (1990) "HIV, AIDS and Intravenous Drug Use: Some Considerations," Journal of Drug Issues, Spring, 20(2):181-194.

Inciardi, J. and Page, J. (1991) "Drug Sharing among Intravenous Drug Users," AIDS, 5(6):772-773.


Lofland, J. (1971) Analyzing Social Settings. Belmont, California: Wadsworth.


Maddux, J. and Desmond, D. (1975) "Reliability and Validity of Information from Chronic Heroin Users." Journal of Psychiatric Research 12: 87-95.

Magura, S., Grossman, J., Lipton, D., et al. (1989) "Determinants of Needle Sharing among Intravenous Drug Users," American Journal of Public Health, 79:459-462.

Marmor, M., Des Jarlais, D., Cohen, H., et al. (1987) "Risk Factors for Infection with Human Immunodeficiency Virus among Intravenous Drug Abusers in New York City," AIDS, 1:39-44.

Marmor, M., Des Jarlais, D., Friedman, S., Lyden, M. and El-Sadr, W. (1984) "The Epidemic of Acquired Immunodeficiency Syndrome (AIDS) and Suggestions for its Control in Drug Abusers," Journal of Substance Abuse Treatment. 1:237-247.


Nurco, D. (1975) "Studying Addicts Over Time: Methodology and Preliminary Findings," American Journal of Drug and Alcohol Abuse 2: 183-196.


Olesen, V. and Whittaker, E. (1968) "Conditions Under Which College Students Borrow, Use and Alter Slang," American Speech, Vol. XLIII, 3, October.

Ottomanelli, G., Kramer, T., Bihari, B., Fine, J., Heller, S. and Mosely, J. (1990) "AIDS-Related Risk Behaviors among Substance Abusers," The International Journal of Addictions, 25(3):291-299.


Page, J., Smith, P. and Kane, N. (1991) "Shooting Galleries, Their Proprietors, and Implications for Prevention of AIDS," Drugs and Society, 5(1-2):69-85.

Patton, M. (1987) How To Use Qualitative Methods in Evaluation. Beverly Hills: Sage Publications.

Personal Communication, 1992. Prevention Point Volunteers.

Suggested Citation:"EVALUATION METHODS." Institute of Medicine and National Research Council. 1994. Proceedings--Workshop on Needle Exchange and Bleach Distribution Programs. Washington, DC: The National Academies Press. doi: 10.17226/4552.
×

Rhodes, F., Corby, N., Wolitski, R., Tashima, N., Cathleen, C., Yankovich, D. and Smith, P. (1990) "Risk Behaviors and Perceptions of AIDS among Street Injection Drug Users," Journal of Drug Education, 20(4):271-288.

Rossi, P. and Freeman, H. (1989) Evaluation: A Systemic Approach. Beverly Hills: Sage Publications.


Schatzman, L. and Strauss, A. (1973) Field Research: Strategies for a Natural Sociology. Englewood Cliffs, NJ: Prentice Hall.

Schoenbaum, E., Hartel, D., Selwyn, P., et al. (1989) "Risk Factors for Human Immunodeficiency Virus Infection in Intravenous Drug Users," The New England Journal of Medicine, 321:874-879.

Selwyn, P., Feiner, C., Cox, C., et al. (1987) "Knowledge about AIDS and High-Risk Behavior among Intravenous Drug Users in New York City," AIDS, 1:247-254.

Sorensen, J., Guydish, J., Costantini, M. and Batki, S. (1989) "Changes in Needle Sharing and Syringe Cleaning among San Francisco Drug Abusers," New England Journal of Medicine, 320:807.

Spradley, J. (1979) The Ethnographic Interview. New York: Holt, Rhinehart and Winston.

Stephens, R. (1972) "The Truthfulness of Addict Respondents in Research Projects," The International Journal of the Addictions 7(3):549-558.

Stephens, R., Feucht, T. and Roman, S. (1991) "Effects of an Intervention Program on AIDS-Related Drug and Needle Behavior Among Intravenous Drug Users," American Journal of Public Health, 81(5):568-571, May.

Strauss, A. and Corbin, J. (1990) Basics of Qualitative Research. Newbury Park: Sage Publications.


Watters, J. and Biernacki, P. (1989) "Targeted Sampling: Options for the Study of Hidden Populations," Social Problems. 36(4):416-430.

Suggested Citation:"EVALUATION METHODS." Institute of Medicine and National Research Council. 1994. Proceedings--Workshop on Needle Exchange and Bleach Distribution Programs. Washington, DC: The National Academies Press. doi: 10.17226/4552.
×

OPERATIONAL MODELING OF NEEDLE EXCHANGE PROGRAMS

Edward H. Kaplan

1. Introduction and Motivation

Needle exchange is advocated by many as an intervention to slow the spread of HIV transmission among drug injectors via needle sharing (Des Jarlais and Friedman, 1992; Stimson, 1989). Whether needle exchange achieves this end is difficult to determine via direct observation. Repeated HIV testing of injecting drug users (IDUs) participating in outreach programs is rare relative to the number of outreach programs that exist (for an exception see Ljungberg et al., 1991), while anything approaching a controlled trial pitting the HIV incidence rate among IDUs participating in a needle exchange against incidence in a provably equivalent group of IDUs not exchanging needles is unheard of. Pragmatically speaking, this is not a situation that is likely to change (Des Jarlais and Friedman, 1992).

Given the above difficulties, many researchers have turned to behavioral studies of needle exchange participants. Such studies rely on the presence (or absence) of self-reported changes in risky behaviors (such as sharing frequencies and injection rates) as evidence of the impact of needle exchange (Donoghoe et al., 1989; GAO, 1993; Hagan et al., 1991; Hartgers et al., 1989; Joseph and Des Jarlais, 1989; Watters and Cheng, 1991). Social desirability effects might play a role in skewing self-reported behavioral data. For example, in a program environment where participants are clearly instructed not to share needles, what should one expect in response to a question like "How often do you share needles?" This in itself does not prove that self-reported data are inaccurate. It does mean, however, that it is hard to know the degree to which self-reported changes in behavior reflect the truth.

This paper offers a rather different set of ideas for evaluating needle exchange programs. These methods have been described elsewhere as they were developed (Kaplan 1994; Kaplan and Heimer, 1993, 1994), but it seems useful

Suggested Citation:"EVALUATION METHODS." Institute of Medicine and National Research Council. 1994. Proceedings--Workshop on Needle Exchange and Bleach Distribution Programs. Washington, DC: The National Academies Press. doi: 10.17226/4552.
×

to bring their details together in one place. The common thread in the techniques to be described is the utilization of mathematical models of needle exchange operations in conjunction with objectively observable data such as the volume of needles distributed, the dates of program visitation, the circulation time of needles, and the fraction of returned needles that test positive for HIV. The models lead to testable conjectures that a priori could lead to either positive or negative conclusions regarding the effectiveness of needle exchange, while the data enable the statistical testing of these conjectures.

Two different modeling approaches will be described below. The first of these attempts to estimate the incidence of HIV infection among participating drug injectors using nothing other than data describing client visits and the measured level of infection in needles (Kaplan and Heimer, 1994). The second approach develops an operational theory of needle exchange that not only provides an estimate for the relative impact of the program on HIV incidence among participating IDUs, but also provides a compelling explanation for how the physics of needle exchange might conspire to reduce HIV transmission in the absence of major changes in behavior (Kaplan, 1994; Kaplan and Heimer, 1993). Both of these models have been implemented using data collected in conjunction with the evaluation of the legal program operated by the AIDS Division of the New Haven, Connecticut Department of Health, and the results of these studies will be discussed. Technical details involved in both the derivation and interpretation of these models will be stressed, while suggestions for how one might implement these ideas in other settings will be made where appropriate.

While the focus of this paper is on methodology, it is worth stating the results of the New Haven studies up front. The models strongly suggest that needle exchange has reduced needle-borne HIV infections among program participants from the beginning of the program in November 1990 through June 1992. One model suggests a relative reduction in transmission on the order of 33%, while the other does not allow rejection of the hypothesis that no new infections have occurred among needle exchange participants since the program started.

In July 1992, the purchase of syringes without prescription at Connecticut pharmacies became legal (Valleroy et al., 1993; Weinstein and Hadler, 1993). Coincident with the decriminalization of syringe purchase

Suggested Citation:"EVALUATION METHODS." Institute of Medicine and National Research Council. 1994. Proceedings--Workshop on Needle Exchange and Bleach Distribution Programs. Washington, DC: The National Academies Press. doi: 10.17226/4552.
×

and sale in Connecticut was a large drop in the number of IDUs participating in the New Haven program. As decriminalization changed the environment in which the New Haven needle exchange was operating in a major fashion, most of the data discussed in this paper will be limited to the first 20 months of the New Haven needle exchange.

2. Estimating HIV Incidence Among Needle Exchange Participants (Kaplan and Heimer, 1994)

2.1 An Idealized Research Environment

To begin, imagine an idealized research environment where one can repeatedly test a cohort of m IDUs for HIV infection over time. If the prevalence of HIV infection in the IDUs is denoted by ø and the incidence rate of new infections is given by ρ infections per uninfected IDU per unit time, then the probability that a randomly selected IDU would become infected within a duration of length δ is given by



Pr(Infection)

(1)

for sufficiently small values of the product ρδ (if ρδ < 0.10 then the error in the approximation above is no worse than about 5%). Note that this is a short run perspective, in that it is assumed that both the prevalence δ and the incidence p remain constant over the duration of the study. Let yi = 1 if the ith IDU in the sample becomes infected during δ (this assumes that the specific date of infection is unknown, which is reasonable if people are tested only at the beginning and end of the study period, for example), and yi = 0 otherwise. First, from data one would discover that approximately IDUs in the cohort were infected at the start of the study (for the prevalence of HIV infection was assumed to equal ø). Second, one would reason that based on equation (1) above. The raw incidence rate p would thus be estimated by dividing (actually, one would divide by the product of δ and the observed number of initially uninfected IDUs, but this latter number is roughly

Suggested Citation:"EVALUATION METHODS." Institute of Medicine and National Research Council. 1994. Proceedings--Workshop on Needle Exchange and Bleach Distribution Programs. Washington, DC: The National Academies Press. doi: 10.17226/4552.
×

m(1-ø)). At the end of duration δ, one would have two ways of expressing the incidence rate: the number of new infections per uninfected drug injector per unit time, given by ρ, or the unconditional number of new infections per IDU per unit time, given by θ. It is easy to show that the expression is also the maximum likelihood estimator of θ (again assuming ρ δ is sufficiently small).

2.2 A Needle Exchange Environment Where Only Needles Are Tested

With the above set of computations in mind for future reference, consider now the more difficult research environment of a needle exchange program. Suppose that no data are available regarding the HIV status of any individual IDU participating in the program, for program officials have decreed that HIV testing might dissuade IDUs from joining the needle exchange. However, data are available regarding the HIV status of samples of needles returned to the program (this is in fact the situation in the needle exchange operated by the AIDS Division of New Haven's Health Department). Specifically, suppose that for a given IDU the following data are available: the earliest and latest dates of needle exchange (and hence the duration δ that the IDU was exposed to the program), and a series of indicator variables x1, x2, ..., xn corresponding to n returned needles ordered by date of return. The indicator xi equals one if the ith needle in sequence tests positive, and zero otherwise. What can one infer from such data?

Note that if needles were never shared and all laboratory testing of needles was error-free, the above data record would yield information equivalent to the idealized incidence trial described above. Unfortunately, one expects both laboratory errors and needle sharing to occur. Even in the presence of such distractions, it is possible to estimate statistically whether a given IDU has become infected. This is because of the following observation: whatever the laboratory error rates and sharing patterns are, the likelihood that a returned needle will test positive for HIV after an IDU has become infected should be higher than the likelihood that a needle returned by the same IDU would test positive before the IDU becomes

Suggested Citation:"EVALUATION METHODS." Institute of Medicine and National Research Council. 1994. Proceedings--Workshop on Needle Exchange and Bleach Distribution Programs. Washington, DC: The National Academies Press. doi: 10.17226/4552.
×

infected. One should therefore be able to detect a new infection by observing an upward shift in the fraction of returned needles that test positive.

2.3 A Maximum Likelihood Change Point Model

This intuition can be formalized. Let π(+) (π(-)) denote the probability that needles test positive conditional on their return by an IDU who is in truth infected (uninfected). As this procedure is being applied on an IDU by IDU basis, there is no need for the values of π(+) and π(-) to be common across IDUs. Indeed, these probabilities allow for differences in needle sharing and needle cleaning patterns across IDUs, in addition to pure laboratory testing errors. Suppose that the IDU in question became infected between the sth and s+1st needles tested. The probability of observing the data record x1, x2, ... xs, xs+l, ..., xn is then given by the likelihood function

(2)

The model formulated above is a change point model (Cox, 1970; Page, 1955), and the unknown parameters π(-), π(+), and s can be estimated from the data record x1, x2, ..., xn via the method of maximum likelihood. Conditional on s, it is clear that the maximum likelihood estimators for π(-) and π(+) are given by and unless the former is larger than the latter, in which case the maximum likelihood estimators for both probabilities are given by (for logic insists that π(+) ≥ π(-)). Substituting the above formulas into (2) enables one to conduct a search over s∈(1, 2, 3, ..., n-1) to find that value of s that maximizes the value of in (2) above (actually, one only needs to consider those values of s such that xs = 0 and xs + 1 =1 in the search process, as is easily shown); call the maximized value of the likelihood function .

Suggested Citation:"EVALUATION METHODS." Institute of Medicine and National Research Council. 1994. Proceedings--Workshop on Needle Exchange and Bleach Distribution Programs. Washington, DC: The National Academies Press. doi: 10.17226/4552.
×

Armed with the maximum likelihood estimates for the three change point parameters, it is possible to test the null hypothesis that p(-) = p(+) = p, that is, no new infection occurred (either because the IDU was already infected, or because the IDU was uninfected and remained so over the study period). Under the null hypothesis, the maximum likelihood estimator for p is given by . Substituting this value for both p(-) and p(+) in (2) yields the maximized likelihood under the null hypothesis; call this .

A threshold test can be established as follows: reject the null hypothesis (and conclude that the IDU became infected) if the ratio exceeds some threshold. Specifically, the null hypothesis will be rejected if the quantity 2 exceeds some cutoff denoted by c (this is equivalent to requiring to exceed ec/2).

2.4 Statistical Errors and the Change Point Test

Note that even with a testing procedure to determine whether an IDU became infected solely on the basis of needle testing, it is possible that the procedure suggested will lead to erroneous decisions. However, it is also possible to compute the probabilities that such statistical errors will occur. Specifically, let a denote the probability of concluding that an IDU has become infected given no infection actually occurred (a Type I error), and ß denote the probability of concluding that an IDU has not become infected given that infection actually occurred (a Type II error). How large might these error probabilities be?

To answer this question requires considering all possible needle testing sequences of size n that could occur given true underlying values for the parameters p(-), p(+), and s. There are 2n possible binary sequences, each one representing a different ordered pattern of positive and negative needle tests. These sequences can be indexed by their binary representation so that in a particular sequence , the ith element if and only if

(3)

Suggested Citation:"EVALUATION METHODS." Institute of Medicine and National Research Council. 1994. Proceedings--Workshop on Needle Exchange and Bleach Distribution Programs. Washington, DC: The National Academies Press. doi: 10.17226/4552.
×

for , 1, ..., 2n-1; otherwise (note that for a fixed value of , equation (3) has a unique solution). In sequence , the ith needle tests positive if , otherwise the ith needle tests negative.

Now, suppose that the data from sequence were subjected to the change point test of Section 2.3. The null hypothesis of no new infection would be rejected if 2 exceeds the cutoff c. Let be the indicator variable that equals 1 if the cutoff is exceeded (and thus the change point test leads to rejection of the null hypothesis), and 0 otherwise. The overall probability of rejecting the null hypothesis for the change point test is then given by

(4)

Equation (4) enables the calculation of the exact probability of rejecting the null hypothesis conditional on the true values of the underlying parameters in the model. For example, suppose that in truth no new infection has occurred, and the probability of any needle testing positive is given by p. The probability of obtaining a particular sequence in this circumstance would then equal

(5)

Substitution of equation (5) into (4) above then enables the calculation of a for any given sample size n, underlying needle prevalence p, and cutoff criterion c.

Of interest is the following robustness result obtained numerically. Given employment of the threshold criterion 2 , asymptotic maximum likelihood theory might tempt one to relate c to the 1-a percentage point of the X2 probability distribution with 2 degrees of freedom (Cox and Hinkley, 1974), for the difference in the number of parameters in the change point model (3: p(+), p(-), and s) and the null hypothesis model (1: p) equals 2. This would not be correct, for the likelihood function of equation (2) is not differentiable with respect to s. However, equations (4) and (5) enable exact computation of the significance level a for

Suggested Citation:"EVALUATION METHODS." Institute of Medicine and National Research Council. 1994. Proceedings--Workshop on Needle Exchange and Bleach Distribution Programs. Washington, DC: The National Academies Press. doi: 10.17226/4552.
×

comparison with the X2 result. The reason for seeking such an approximation is that while the exact computations for a (and ß) are possible, they are tedious (for example, a sample of n = 15 needles gives rise to 215 = 32,768 different sequences, each one of which must be passed through the change point procedure to determine the value of . The 95th percentile of the X2 distribution with 2 degrees of freedom is given by 5.991, so setting c = 5.991 should yield values of a close to 0.05 if the X2 provides reasonable guidance. As reported in Kaplan and Heimer (1994), for sample sizes (n) of 8, 10, 12, and 15, and assumed values of p running from 0.10 through 0.90 in increments of 0.10, the exact significance levels computed ranged from 0.0496 through 0.0663. Thus, even though the use of the X2 to select cutoff values is not justified on the theoretical grounds of asymptotic likelihood theory, its use does appear reasonable from the (limited) numerical results obtained. This is of course important, for in any true application of the change point test, one does not know the underlying true value of p (assuming the null hypothesis is correct), while one will also be faced with needle testing sequences of different lengths.

The power of the change point test can also be explored using equation (4) above. Details are provided in the appendix of Kaplan and Heimer (1994). In the numerical computations reported there, the power of a change point test using a cutoff of c = 5.991 for samples of size 8, 10, 12 and 15 assuming p(-) = 0.15 and p(+) = 0.85 was explored as a function of s, the true change point parameter. With the exceptions of s values equal to 1 or n-1, the power obtained was close to or exceeded 50%. For values of s in the middle of the sample (s » n/2), the power was close to or exceeded 70%.

It therefore seems safe to conclude that for most applications, applying a cutoff of c = 5.991 to the change point test described in Section 2.3 results in a Type I error probability of a a » 0.05, and a Type II error probability of ß = 0.50. These quantities enable the construction of a procedure for estimating HIV incidence among needle exchange participants based solely on the testing of returned needles.

Suggested Citation:"EVALUATION METHODS." Institute of Medicine and National Research Council. 1994. Proceedings--Workshop on Needle Exchange and Bleach Distribution Programs. Washington, DC: The National Academies Press. doi: 10.17226/4552.
×
2.5 Estimating HIV Incidence from Change Point Test Results

To summarize the discussion thus far, a statistical model has been proposed for determining whether or not an individual IDU has become infected based solely on the results of HIV tests on returned needles. The modeling also allows the computation of the statistical significance (α) and power (1-β) of the test, so the likely size of statistical errors can be known in advance. Now, recall from equation (1) that the probability that an IDU becomes infected over some duration δ approximately equals θδ, where θ is the unconditional incidence rate, that is, the number of new infections per IDU per unit time. Instead of observing individual tests on IDUs, however, imagine observing the results of change point tests conducted on the sequences of needles returned by program participants. What is the probability that one would conclude an IDU became infected on the basis of a change point test?

Clearly there are two possibilities. First, the IDU in question really became infected (with probability θδ), and the change point test had the power to detect this new infection (with probability 1-β). Alternatively, the IDU did not become infected (with probability 1-θδ), but the change point test committed a Type I statistical error, and falsely concluded that the IDU became infected (with probability α). Therefore, the probability r(θ) that one would conclude an IDU became infected on the basis of a change point test is given by

r(θ) = θδ(1-β) + (1-θδ)α = α + (1-α-β)θδ. (6)

By analogy with Section 2.1, let yi = 1 if the change point test performed on the sequence of needles returned by the ith IDU suggests that infection has occurred (i.e. 2 for the ith sequence of needles), and yi = 0 otherwise, i = 1, 2,..., m. A very simple test of the hypothesis that no new infections have occurred among the IDUs returning the needles can be constructed by setting θ=0 (as required by the hypothesis of no new infections). In this case, r(θ) = α, thus the total number of rejected change point tests follows the binomial probability

Suggested Citation:"EVALUATION METHODS." Institute of Medicine and National Research Council. 1994. Proceedings--Workshop on Needle Exchange and Bleach Distribution Programs. Washington, DC: The National Academies Press. doi: 10.17226/4552.
×

distribution with parameters m and a. It is then an elementary task to compute the probability that at least as many rejected change point tests as actually observed would occur solely due to chance.

It is also possible to estimate θ via maximum likelihood. Given δi, the duration of exposure of the ith IDU to the needle exchange as discerned from the dates of first and last program visits (or perhaps more appropriately, from the dates of first and last visits from which returned needles were tested for HIV), the probability of observing a rejected change point test follows equation (6) after substituting δi for δ; call this probability ri(θ). As a consequence, the probability of observing the particular set of change point results for all m IDUs is given by

(7)

Maximizing over θ subject to the constraint that θ ≥ 0 can be achieved directly. The maximum likelihood estimator will equal zero for any change point significance level α;  otherwise, is the unique root of the equation

(8)

Note that if δi= δ for i = 1, 2, . . ., m, then equation (8) yields a simple formula for , given by

(9)

In the idealized world of perfect statistical testing, both a and β would equal zero, yielding , the same result cited in Section 2.1.

Suggested Citation:"EVALUATION METHODS." Institute of Medicine and National Research Council. 1994. Proceedings--Workshop on Needle Exchange and Bleach Distribution Programs. Washington, DC: The National Academies Press. doi: 10.17226/4552.
×
2.6 Application to the New Haven Needle Exchange: The Syringe Tracking and Testing System and Statistical Results

The data structures alluded to above have been realized as part of the evaluation of New Haven's needle exchange program. The unique syringe tracking and testing system (or STT) implemented in New Haven works as follows (Kaplan and O'Keefe, 1993): all participants in the program receive a code name of their own choosing. In addition, all needles distributed by the program are endowed with a unique tracking code. When the program started, these codes were placed by hand with a paint pen directly on outbound syringes, and covered with transparent tape for protection (non-detachable needle/syringe pairs are distributed by the program, so the terms needle and syringe are used interchangably). Since the fall of 1992 syringes have been tracked by bar code. At the time exchange transactions occur, outreach workers record the date, location, and code name of the person receiving the needles. When needles are returned to the program, the date, location, and code name of the person returning the needles are also recorded. Consequently, a complete record of syringe transactions and client participation is compiled.

In addition to syringe tracking, samples of returned needles are tested for the presence of HIV-1 proviral DNA using polymerase chain reaction (PCR) techniques developed by Robert Heimer (see Heimer et al. (1992a)). Recognizing that a simple random sample of returned needles would be weighted heavily to reflect those who exchange most often, a two-stage sampling procedure has been implemented to achieve greater client coverage. Needles are exchanged in batches of at most five. In the first sampling stage, all syringes are retrieved from returned batches of at most two, while two syringes are taken from batches of size three through five. The second stage systematically samples every 10th needle selected in the first stage. From November 1990 through June 1992, this procedure selected 2,813 of the 49,405 needles distributed for testing.

The change point models were implemented on a subset of the needle testing data reported above. First, testing results were available only for needles returned by the end of May 1992 at the time this study was undertaken. Second, only those clients who had returned at least five tested needles were considered for analysis, recognizing the inability of

Suggested Citation:"EVALUATION METHODS." Institute of Medicine and National Research Council. 1994. Proceedings--Workshop on Needle Exchange and Bleach Distribution Programs. Washington, DC: The National Academies Press. doi: 10.17226/4552.
×

the change point test to provide meaningful results for smaller values of n. The STT provided a total of 1,920 needles representing 132 different clients with at least five tested needles (averaging 14.5 tested needles per client). Just over 42% of these needles tested HIV positive via PCR. Exposure times were taken as the time between the first and last dates tested needles were returned inclusive. The total derived exposure time was 34,903 person days in the needle exchange, averaging 264 days per client.

Of the 132 change point tests conducted, six rejected the null hypothesis of no infection using the cutoff c = 5.991. If one was to simply divide this number by the total exposure time, one would estimate an unconditional HIV incidence rate of 6.3 new infections per 100 needle exchange clients per year. However, this ignores the chance errors one can expect from applying the change point test.

Having observed that , a Type I error probability of a = 0.05 associated with the change point test (as suggested by numerical studies) easily accounts for the data. Under the null hypothesis that no new infections occurred (that is, that θ = 0), the probability of observing six or more rejected change point tests is given by the binomial tail probability

(10)

Given that one has more than a 65% chance of observing the change point results assuming that θ = 0, it is not possible to reject the hypothesis that in the New Haven needle exchange, no new infections have occurred among those IDUs studied.

The maximum likelihood approach supports this point as well. The value of . For any a ≥ 0.0398, θ = 0, and as numerical studies of the change point test suggest that a is in the neighborhood of 0.05, it is indeed the case that .

It is worth mentioning at this point that these results were not in any way preordained. It could have been the case that the change point test rejected the null hypothesis for a large percentage of those clients studied. It is important to note that at the conventional significance level of 5%, one would not be able to reject the null hypothesis of no new infections unless 12 or more change point tests suggested that infections

Suggested Citation:"EVALUATION METHODS." Institute of Medicine and National Research Council. 1994. Proceedings--Workshop on Needle Exchange and Bleach Distribution Programs. Washington, DC: The National Academies Press. doi: 10.17226/4552.
×

had occurred. Given that only half that number was actually observed suggests that in the New Haven needle exchange program, the incidence rate of new infections is not high. Furthermore, since more than 42% of the needles tested HIV positive, it is clear that this population had a positive HIV incidence rate in the past.

While the change point model provides an approach to estimating HIV incidence among needle exchange participants, it does not explain how needle exchange operations might be expected to effect HIV transmission rates. Doing so requires deeper understanding of the physics of needle exchange. Models describing the physical workings of needle exchange and implications for disease transmission are described next.

3. The Physics of Needle Exchange (Kaplan, 1994)

3.1 A Qualitative Overview

This section of the paper formalizes a circulation theory of needle exchange developed over the past two years. Before presenting the mathematical and statistical details of this theory, however, it is useful to summarize the intuition underlying the key ideas. The mathematical arguments that follow provide a rigorous mechanism for expressing and testing this intuition.

To begin, try to imagine the HIV status of a needle from the moment it is introduced into a population of drug injectors. Many different experiences are possible, but at any moment in time a needle may be viewed as either contaminated or not. Needles can only become infected following use by an HIV infected IDU, though not all needles necessarily become contaminated following such use (also, effective post-injection bleaching may render some needles non-infectious in spite of having been used by infected IDUs). Infected needles can become decontaminated for a variety of reasons, including effective pre-injection bleaching (though not all bleaching is effective), dilution of infectious blood with the blood of uninfected IDUs following several shared injections, or natural inactivation

Suggested Citation:"EVALUATION METHODS." Institute of Medicine and National Research Council. 1994. Proceedings--Workshop on Needle Exchange and Bleach Distribution Programs. Washington, DC: The National Academies Press. doi: 10.17226/4552.
×

of the virus. Whatever the relative contamination and decontamination rates are per needle, clearly the longer a needle remains in circulation, the higher the likelihood that the needle in question becomes infected.

Aggregating the stochastic infection processes for individual needles across all needles in the population leads to an average level of infection among all needles at any point in time. This average level may be interpreted as the probability that a needle selected at random is contaminated with HIV. In addition to the relative contamination and decontamination rates, the average level of infection depends on the distribution of needle circulation times, with stochastically longer circulation times associated with higher average levels of infection in the needles.

Needle exchange may be thought of as a mechanism that operates directly on the distribution of needle circulation times (Kaplan and Heimer, 1993). First, note that by exchanging needles, the total number of needles in circulation does not change in theory for a law of conservation of needles applies: the rate with which needles are distributed is balanced by the rate with which needles are returned. To the extent that one believes that HIV transmission among IDUs is a function solely of the number of needles in circulation, then, one should not expect needle exchange to change matters much. Instead, what one achieves by exchanging needles is the interruption of the needle circulation process. This stochastically reduces the length of time for which needles remain in the population. The greater the rate with which needles are exchanged, the greater the reduction in needle circulation times. This is the key operational link between needle exchange programs and HIV transmission, for reducing needle circulation times acts via the stochastic infection process to lower the average level of infection in the population of needles. The intuition is simple: if needles are available for shorter periods of time per needle, then the likelihood that different IDUs will use the same needle declines. In effect, needle exchange causes needles to share fewer people.

Reducing the mean level of infection in circulating needles is important, for those who persist in sharing needles are in effect sampling from the population of needles in circulation. If the mean level of infection in needles declines, then the chance that a person who persists in sharing needles will encounter a contaminated needle also declines. Since

Suggested Citation:"EVALUATION METHODS." Institute of Medicine and National Research Council. 1994. Proceedings--Workshop on Needle Exchange and Bleach Distribution Programs. Washington, DC: The National Academies Press. doi: 10.17226/4552.
×

one can only become infected via needle sharing from contaminated needles, it follows that the incidence rate of HIV infection by needle sharing will be proportional to the average level of infection in circulating needles. The relative success of a needle exchange in reducing HIV transmission can therefore be measured by the relative drop in the level of infection in circulating needles.

The theory summarized above does not imply that needle exchange is an instant success. To be effective in reducing HIV incidence, the theory suggests that exchange programs must exchange needles with sufficient rapidity to reduce circulation times by a large amount. Another important point is that this theory offers no support for the contention that needle exchange reduces HIV transmission among anyone not participating in the program. Needle exchange might cause a reduction in such transmission (perhaps via averting sexually transmitted infections to non-injecting sex partners of client IDUs, or via the placement of non-client IDUs in drug treatment programs), but the analysis developed here does not consider such possibilities. The fair interpretation of what follows is that the physics of needle exchange alone can conspire to confer protective benefits to those who regularly exchange needles.

The validity of the theory can be tested providing the requisite data are available to do so. Such data have been assembled via New Haven's syringe tracking and testing system. Less exhaustive approaches to data collection could also yield sufficient information for applying the ideas presented below, as will be discussed briefly later on.

With the above intuition firmly in mind, it is time to explore the physics of needle exchange.

3.2 The Needle Infection Process

As described above, circulating needles are always in one of two states, HIV-contaminated or uncontaminated. Aiming for simplicity, a two-state continuous time Markov process is proposed as a model for the dynamics of needle infection. Uncontaminated needles are assumed to become infected with rate λ per uncontaminated needle per unit time, while HIV-contaminated needles become uncontaminated with rate µ per contaminated needle per unit

Suggested Citation:"EVALUATION METHODS." Institute of Medicine and National Research Council. 1994. Proceedings--Workshop on Needle Exchange and Bleach Distribution Programs. Washington, DC: The National Academies Press. doi: 10.17226/4552.
×

time. The perspective is again short term in that the epidemiological and biological processes that contribute to λ and µ as described in Section 3.1 are assumed to remain stable. Thus, the prevalence of infection (ø), needle injection, sharing and cleaning rates, and viral inactivation rates are all assumed constant over the period of study, leading to stable values of λ and µ. Let π(t) denote the probability that a needle that has been circulating for t time units is HIV-contaminated. Following standard Markov process theory (Bhat, 1984), π(t) can be found from the recursion

π(t + △t) = (l-π(t)) λ △t + π(t) (1-µt) + o(△t). (11)

Letting △t → 0 in equation (11) leads to the first order linear differential equation

(12)

which solves to yield

(13)

where π0 is the probability that the needle is already HIV-contaminated when introduced to the population (and should equal zero if the needle in question was provided via a needle exchange program). Equation (13) quantifies the intuition described earlier, in that the probability that a needle is HIV-contaminated increases from π0 to the equilibrium value of λ/(λ+µ) with increasing circulation time.

Suggested Citation:"EVALUATION METHODS." Institute of Medicine and National Research Council. 1994. Proceedings--Workshop on Needle Exchange and Bleach Distribution Programs. Washington, DC: The National Academies Press. doi: 10.17226/4552.
×
3.3 The Needle Circulation Process

Equation (13) describes the probability that a needle is HIV-contaminated following a fixed circulation time of duration t. However, there are many needles circulating at any moment in time. Of interest is the mean level of infection in those needles circulating at any moment in time, that is, the probability that a randomly selected circulating needle is infected.

To compute this probability requires a model of the needle circulation process. Let random variable T denote the complete length of time for which a needle remains in the population, and random variable TR denote the complete length of time for which a circulating needle selected at random remains in the population. The random variable TRis thus obtained by length biased sampling or random incidence (Cox, 1962; Drake, 1967), for the likelihood of finding a complete circulation interval of a given duration is proportional to both the frequency with which such intervals occur and the duration of the interval itself. The probability density of random variable TR is related to the density of random variable T by the equation

(14)

where E(T) denotes the expected circulation time, and probability densities are denoted by f.

It is now necessary to obtain the probability density function for the elapsed circulation time of a needle selected at random from the population. Note that conditional upon selecting a needle with a complete circulation interval equal to, say, u, the elapsed circulation time will simply be distributed uniformly between 0 and u. Therefore, the probability density function for the elapsed circulation time of a needle selected at random (elapsed circulation time is denoted by random variable TE) is given by

(15)

Suggested Citation:"EVALUATION METHODS." Institute of Medicine and National Research Council. 1994. Proceedings--Workshop on Needle Exchange and Bleach Distribution Programs. Washington, DC: The National Academies Press. doi: 10.17226/4552.
×

where FT(t) is the cumulative distribution function associated with the original circulation time distribution.

The probability that a circulating needle selected at random is HIV-contaminated, , is therefore equal to

(16)

Equation (16) is a general result, in that it shows how to obtain the average level of infection in circulating needles for any needle infection process (not just the Markov process considered earlier) and for any needle circulation process. However, in studies conducted thus far, only the Markov infection process has been considered in concert with an exponential circulation time probability density function given by

(17)

where τ is the mean circulation time. The exponential possesses the special property that the distribution of elapsed circulation times equals the original distribution of circulation times (the so-called memoryless property). The average level of infection in circulating needles for the Markov-exponential model, denoted by (τ), is thus given by

dt

(18)

Equation (18) again has the correct intuitive properties. Note that as τ approaches 0, approaches π0 as it should, while as t becomes very large, approaches the equilibrium value λ/(λ+µ).

Suggested Citation:"EVALUATION METHODS." Institute of Medicine and National Research Council. 1994. Proceedings--Workshop on Needle Exchange and Bleach Distribution Programs. Washington, DC: The National Academies Press. doi: 10.17226/4552.
×

Having established a model describing the average level of infection in circulating needles at a random moment in time, it is possible to consider how needle exchange effects this mean level of infection. This is the subject of the next section.

3.4 Needle Exchange Reduces Circulation Times

Needle exchange physically interrupts the needle circulation process by replacing circulating syringes with new ones every time an exchange transaction occurs. Thus, the physical result of needle exchange is to shorten the circulation times of needles. A specific hypothesis is offered for the nature of the relationship between needle exchange and circulation times. Let v denote the rate with which needles are exchanged per IDU per unit time, and τ(v) denote the mean circulation time that results when needles are exchanged with rate v. The mean removal rate of needles from the population when a program with exchange rate v is in force will be denoted by ξ(v). Clearly ξ(v) = 1/τ(v) by definition.

In the absence of needle exchange (v=0), there will still be some baseline removal rate (ξ(0)) and mean circulation time (τ(0)) in the population of needles. However, as n increases, one expects ξ( v) to increase as well. In particular, let

ξ(v) = a + bv (19)

represent the impact of needle exchange on the removal rate of needles (the parameters a and b are both non-negative). The parameter a denotes the baseline removal rate (so τ(0)=1/a), while b may be interpreted as a transfer coefficient converting needle distribution per client to needle removal per circulating needle. The coefficient b accounts for the difference in the population size of clients and the population size of needles (the greater the ratio of clients to needles, the greater the value of b). In addition, b allows for the possibility of substitution effects,

Suggested Citation:"EVALUATION METHODS." Institute of Medicine and National Research Council. 1994. Proceedings--Workshop on Needle Exchange and Bleach Distribution Programs. Washington, DC: The National Academies Press. doi: 10.17226/4552.
×

in that the availability of needles from an exchange program may partially substitute for needles previously obtained from other sources (the greater the degree of substitution, the lower the magnitude of b).

The models above present a compact theory of the physical effect of needle exchange. To recap: implementing a needle exchange program with an exchange rate of v needles per client per unit time is hypothesized to increase per needle removal rates (ξ(v)) in accordance with equation (19). This reduces the average circulation time among circulating needles, for τ(v) equals 1/ξ(v). Reducing τ(v) reduces in accordance with equation (18).

As HIV transmission via needle sharing requires injection with a contaminated needle, it is reasonable to consider the relative reduction in the mean level of HIV infection in circulating needles as a measure of the relative reduction in HIV incidence. The key evaluation performance measure to be considered is therefore equal to

(20)

To compute absolute reductions in incidence requires an estimate of the baseline HIV incidence rate before needle exchange began. Such estimates are not available via the theory advanced here, but can be obtained from other back-of-the-envelope calculations to be demonstrated shortly if actual observations from field studies are not available (which will most often be the case).

3.5 Back-of-the-Envelope Estimates of Baseline HIV Incidence Rates

If one wishes to estimate the actual number of infections prevented via a needle exchange program, one requires estimates of baseline HIV incidence among those IDUs participating in the program. This in itself is a difficult task as noted in Section 1. What follows are some very simple suggestions for producing ballpark estimates of baseline HIV incidence. The

Suggested Citation:"EVALUATION METHODS." Institute of Medicine and National Research Council. 1994. Proceedings--Workshop on Needle Exchange and Bleach Distribution Programs. Washington, DC: The National Academies Press. doi: 10.17226/4552.
×

idea is not to be precise, but rather to develop a range of values that can be combined with equation (20) to produce upper and lower bounds on the actual number of infections averted by needle exchange.

The first suggestion borrows from an established result in epidemiology: for any disease in equilibrium (i.e. steady state), the prevalence of the disease equals the product of the incidence rate and the mean duration of infectiousness (Mausner and Kramer, 1985). As a note of independent interest, this result is actually of much greater generality. For example, in the theory of stochastic service systems (better known as queueing theory), the mean number of customers in the system equals the product of the customer arrival rate and the average sojourn time in the system for any workload conserving system in equilibrium, a result known as Little's Theorem (Gross and Harris, 1985). One sees the analogy immediately: new infections correspond to arriving customers, prevalent infections correspond to customers in the system, and the duration of infectiousness corresponds to the sojourn time in the system.

Queuing theory aside, one can invoke the prevalence/incidence law if one believes that the HIV epidemic has become (locally) endemic among drug injectors. Following our notation of Section 2, an estimate of the unconditional incidence rate θ is given simply by

θ =ø γ

(21)

where ø is the prevalence of HIV infection, and γ is the mean progression rate, and can be thought of as the reciprocal of the duration of infectiousness. In applying this relationship, ''duration of infectiousness" means the length of time during which an infectious IDU is behaving in a manner that is capable of spreading disease. The usual assumption among AIDS epidemic modelers is to set γ equal to the reciprocal of the mean time from HIV infection through the development of AIDS symptoms (Anderson and May, 1991; Kaplan, 1989). While it is possible that IDUs with frank AIDS may continue to share needles or engage in unprotected sexual activity, it is also possible that HIV-infected IDUs cease risky behavior before complete progression to AIDS.

Suggested Citation:"EVALUATION METHODS." Institute of Medicine and National Research Council. 1994. Proceedings--Workshop on Needle Exchange and Bleach Distribution Programs. Washington, DC: The National Academies Press. doi: 10.17226/4552.
×

There are some caveats that must be applied to equation (21). First, this relation is only sensible if one does believe that the epidemic has achieved an equilibrium level of infection. This may be true for drug injectors in some areas of the United States such as the northeastern seaboard (i.e. New York City and environs). Second, not all drug injectors are infected via needle sharing, and it is only such infections that the physics of needle exchange can prevent. While it is not known precisely what percentage of new infections among IDUs are sexually acquired (Nelson et al., 1992; Schoenbaum et al., 1989), some researchers have produced estimates as high as 40% (Dr. James Kahn, personal communication). Therefore, it is reasonable to discount the result of equation (21) to account for possible sexual transmission. A final caveat is that in employing equation (21), one either has a prevalence estimate among program IDUs at baseline, or one uses prevalence information from other sources. If the prevalence value used is a community value, then there may be selection effects of unknown form if those participating in the needle exchange are not representative of all IDUs in the relevant population. Some might argue that those likely to be engaged in a needle exchange are those most likely to look after their own health, an argument for reducing the prevalence among program clients relative to a community value. Alternatively, others might note that needle exchanges seem to attract older drug injectors, an argument for raising the prevalence relative to a community value. Rather than systematically attempting to sort these biases out, the suggestion offered here is to use equation (21) to create a range of plausible values for which credible arguments exist. An example of this will be illustrated later.

A second suggestion for estimating baseline incidence involves the application of the well-known backcalculation technique for reconstructing aggregate HIV incidence data from reported AIDS incidence data corrected for reporting delays (Brookmeyer, 1991; Brookmeyer and Gail, 1988). Backcalculation employs the consensus relationship between HIV infection and AIDS in the form of a convolution integral. Letting denote (lag corrected) AIDS incidence at time t (measured in the number of cases per unit time), denote the HIV incidence rate at time t (measured in the number of infections per unit time), I denote the random variable

Suggested Citation:"EVALUATION METHODS." Institute of Medicine and National Research Council. 1994. Proceedings--Workshop on Needle Exchange and Bleach Distribution Programs. Washington, DC: The National Academies Press. doi: 10.17226/4552.
×

representing the time needed to progress from HIV infection through AIDS, and fI(·) denote the probability density function of I (assumed known), the backcalculation model can be represented as

(22)

The logic of backcalculation is simple: in order for AIDS cases to have been reported at time t, how many HIV infections must have occurred at earlier points in time?

Backcalculation can be applied to AIDS incidence data specific to drug injectors to obtain a rough idea of HIV incidence rates in that population. It is well known that backcalculation offers little information about recent HIV incidence (for the nature of the incubation time distribution precludes the development of AIDS cases shortly after HIV infection occurs). However, one can still develop an idea about the magnitude of HIV incidence from backcalculation. For example, one might conjecture an HIV incidence function with a constant incidence rate for the past several years, and use backcalculation to obtain an estimate of this incidence rate.

Of course, assuming one has been able to obtain an incidence estimate among IDUs in a given geographical area via backcalculation, it must be remembered that this estimate pertains to the IDU population as a whole, and also accounts for all possible sources of infection. Thus, one first needs to ask what percentage of the incidence estimated can be assigned to needle exchange clients at baseline. If one believes that clients are representative of all IDUs in the community, for example, then one could multiply the incidence rate obtained by the ratio of the number of program participants to the number of IDUs in the community. This of course assumes that one has available an estimate of the number of drug injectors in the community, a number which itself is very hard to come by. The figure resulting from the above scaling might be adjusted up or down depending upon one's beliefs regarding the likelihood of infection among needle exchange clients relative to IDUs at large as discussed above. Also, one should discount the resulting figure to account for HIV infections acquired via sources other than needle sharing.

Suggested Citation:"EVALUATION METHODS." Institute of Medicine and National Research Council. 1994. Proceedings--Workshop on Needle Exchange and Bleach Distribution Programs. Washington, DC: The National Academies Press. doi: 10.17226/4552.
×

Lest the reader begin to think that there are so many uncertainties operating here as to render the above suggestions worthless, recall that the idea is not to produce a precise estimate, but rather to produce a range of plausible values. While one may not know the exact number of IDUs in a city, for example, one can reasonably place bounds on the population size using data sources such as arrest records, drug treatment data, and mortality data (Aldrich et al., 1990; Frischer et al., 1993; Newmeyer, 1988), not to mention common sense (e.g., is it reasonable to believe that more than 5% of a city's population regularly injects drugs?). Again, the idea is to consider a credible range of values that will lead to a range of baseline incidence levels. The approach will be illustrated shortly.

3.6 Application to the New Haven Needle Exchange Program

It is possible to apply the ideas of the past several sections to the evaluation of the New Haven needle exchange program due to the availability of data from the syringe tracking and testing system as described in Section 2.6 above. The sampling procedure for obtaining needles for testing was already described there, and the fraction of tested needles distributed in a given month that subsequently tested HIV positive via PCR is taken as an empirical measure of , the mean level of infection in circulating needles among program clients. As the exchange program will accept any needles in exchange for new ones, there is no reason to believe that those needles returned represent a biased selection of all needles in circulation.

Needle distribution rates per client (v) were defined monthly as the number of needles distributed by the program in a given month divided by the number of clients who visited the program at least once during the month in question. Mean needle circulation times were also indexed by month of distribution. As needles are typically exchanged in batches of at most five per visit, mean circulation times are defined as the average elapsed time between needle distribution and return for returned needles normalized by the average number of needles distributed per batch, to obtain an average circulation time per needle. Such normalization ensures that the

Suggested Citation:"EVALUATION METHODS." Institute of Medicine and National Research Council. 1994. Proceedings--Workshop on Needle Exchange and Bleach Distribution Programs. Washington, DC: The National Academies Press. doi: 10.17226/4552.
×

circulation times attributed to needles individually exchanged four times per week (i.e. in batches of size one) are equivalent to circulation times for needles exchanged weekly in batches of size four, for example.

To gain a feel for the scale of needle exchange operations in the New Haven program, Figure 1 reports the number of distinct client visits, as well as the number of clients who visited the program each month since program inception through June 1993. That the decriminalization of syringe possession without prescription in Connecticut pharmacies apparently had a major impact on participation in the program is evident from this graph. If one focuses on the 20 months prior to July 1992 however, one notes that visitation frequency increased more rapidly than client participation, suggesting that the same clients were exchanging needles more frequently. This in turn suggests that needle circulation times should have fallen over time due to the increasing frequency of exchange.

Figure 2 documents the inbound and outbound flow of needles in the New Haven program over time. Again the apparent impact of syringe decriminalization in July 1992 is visible. Also visible, however, is the close match between the number of needles distributed and returned over time. The closeness of this match is important, for it shows that for the New Haven program, the "law of conservation of needles" is approximately correct. It is clear that there is some leakage in the system. Some of this can be attributed to program policy whereby newly enrolling IDUs with no needles to exchange can receive a single "starter" needle, following which all exchanges are in theory to occur on a one for one basis. However, the amount of leakage that actually occurred exceeds what can be explained by this policy, indicating that in some transactions clients received more needles than they returned as a result of clerical error or deliberate deception.

A major claim of the theory advanced is that more frequent exchanging should lead to a reduction in mean needle circulation times. Focusing now only on the 20 months from November 1990 through June 1992 inclusive, Figure 3 shows that mean circulation times have been reduced. Thus, needle exchange appears to be interrupting the needle circulation process in the manner intended.

Suggested Citation:"EVALUATION METHODS." Institute of Medicine and National Research Council. 1994. Proceedings--Workshop on Needle Exchange and Bleach Distribution Programs. Washington, DC: The National Academies Press. doi: 10.17226/4552.
×

Equation (19) affords another empirical test of the physics of needle exchange. Figure 4 plots ξ(v) (defined monthly as the reciprocal of the mean circulation time) versus the distribution rate v itself. The strength of the linear relationship is clear from the figure. Simple linear regression yields the empirical relation

(23)

Of great interest is the estimated value of , for an estimate of pre-needle exchange circulation times is provided by days. As of the end of July 1992, circulation times had been reduced to between two and three days, so the reduction in needle circulation times appears to be substantial.

Equation (18) proposes a specific expression for the mean level of infection in circulating needles as a function of mean circulation times. Using the circulation time and PCR test results for the first 20 months of the program, it is possible to estimate the unknown parameters π0, λ and µ via maximum likelihood. Specifically, letting nPOS(i) denote the number of needles that tested positive for HIV via PCR in month i, nNEG(i) denote the number of needles that tested negative in month i, and equal equation (18) with mean circulation time equal to (the observed mean circulation time for month i ), the likelihood function constructed is given by

(24)

This function was maximized numerically using the GAUSS maximum likelihood routine (GAUSS, 1992). The resulting maximum likelihood estimates are given by (standard error = 0.028); (standard error = 0.0676); and (standard error = 0.0824).

Suggested Citation:"EVALUATION METHODS." Institute of Medicine and National Research Council. 1994. Proceedings--Workshop on Needle Exchange and Bleach Distribution Programs. Washington, DC: The National Academies Press. doi: 10.17226/4552.
×

A number of results are worth mentioning. First, as all tests were performed on needle exchange needles, one would expect that p0 = 0. This intuition is confirmed statistically, as the maximum likelihood estimate obtained is not significantly different from 0. Second, the fit of the model can be gauged from Figure 5, which plots both observed and modeled mean infection levels in the needles over time. There are four clear outliers that cannot be explained by the theory, but the general fit of the model appears reasonably good.

Third, it is possible to conduct two "holdout" tests of the model. At the beginning of the needle exchange, 160 "street needles" that were delivered to the program in exchange for new needles were selected for HIV testing via PCR. Of these 160 needles, 108 (or 67.5%) tested HIV positive (Heimer et al. 1992b; Heimer et al., 1993). Now, while the true pre-needle exchange circulation times are in truth unknown, equation (23) affords the previously mentioned estimate of 23.5 days. Substituting this into equation (18) with the estimated parameters cited above yields an estimate of equal to 0.6375 (standard error-0.0529 incorporating fluctuations due to the sample size of 160 street needles). That the prediction obtained is in such close accord with the level of infection in the street needles, needles which were not employed in estimating the parameters of equation (18), provides a certain degree of confidence in the model.

The second holdout test involves needles retrieved from an illegal underground needle exchange program that existed prior to the establishment of the legal program. It is interesting (and perhaps surprising) to note that the reach of this program was apparently not great, for among 851 clients of the legal exchange responding to surveys over the course of the last two years, only 12 stated that they formerly obtained needles from underground programs in New Haven (contrasted to more than 70% of such respondents who stated that they purchased needles or obtained them "off the street"). While this statistic is not intended as a precise measure of the fraction of IDUs who used the underground program, it does suggest that this fraction could not have been large. Of 180 needles tested from the underground program, 113 (or 62.8%) tested HIV positive via PCR (Heimer et al. , 1992b; Heimer et al., 1993), extremely close to the prediction of equation (18). It is again important to point out that the needles from the underground program were not used to calibrate the model. A negative

Suggested Citation:"EVALUATION METHODS." Institute of Medicine and National Research Council. 1994. Proceedings--Workshop on Needle Exchange and Bleach Distribution Programs. Washington, DC: The National Academies Press. doi: 10.17226/4552.
×

implication of this result, however, is that it appears that the underground program in question did not achieve a sufficient turnaround of needles to significantly change HIV transmission patterns. This example should again help to convince the reader that the results of these studies are not preordained. Negative results are possible, and occasionally they are obtained as in the example of the underground program.

Returning to the legal needle exchange program, however, it appears that the impact of needle exchange on interrupting the needle circulation process, and hence the HIV transmission process, has been considerable. As the mean circulation time of needles appears to have been reduced to near 3 days, equation (20) suggests a relative reduction in HIV transmission on the order of 33% (obtained by setting t(0) = 23.5 days and . While this sounds impressive, recall that this is only a relative reduction in incidence via needle-borne infections among needle exchange participants. To translate this relative reduction into an actual number of infections averted requires producing a range of baseline incidence estimates.

Consider first the equilibrium method based on equation (21). Prevalence surveys among New Haven drug injectors are not common, but a range of estimates are available. Data from the CDC national seroprevalence surveys included drug injectors sampled at a sexually transmitted disease clinic in New Haven, of whom 35.6% tested HIV positive (CDC, 1990). On the high end, 67% of African American men entering a drug treatment program tested HIV positive (Dr. Patrick O'Conner, personal communication), while early attempts to estimate HIV prevalence among needle exchange participants using more assumption laden models yielded estimates in the neighborhood of 60% (Kaplan and Heimer, 1992). To be comprehensive, baseline HIV prevalence among needle exchange clients will be assumed to lie somewhere between 30% and 70% inclusive.

A number of studies have documented that the mean (or median) time between HIV infection and AIDS is approximately 10 years in duration (Bacchetti and Moss, 1989; Brookmeyer and Goedert, 1989; Freund and Book, 1990; Muñoz et al., 1989). Therefore, the mean progression rate g will be set equal to 0.10 per infected IDU per year. Equation (21) then suggests that aggregate HIV incidence at baseline among IDUs resides somewhere between three and seven infections per 100 IDUs per year.

Suggested Citation:"EVALUATION METHODS." Institute of Medicine and National Research Council. 1994. Proceedings--Workshop on Needle Exchange and Bleach Distribution Programs. Washington, DC: The National Academies Press. doi: 10.17226/4552.
×

However, not all of these infections would have been transmitted via needle sharing. A number of researchers have documented that drug injectors do become infected sexually (Nelson et al., 1992; Schoenbaum et al., 1989), and some researchers have pegged the fraction of all infections due to sexual transmission among IDUs as high as 40% (Dr. James Kahn, personal communication). For now, it will be assumed that 30% of all infections among drug injectors are acquired sexually, reducing the baseline incidence of needle-borne HIV infections to between 2.1 and 4.9 per 100 IDUs per year. The New Haven data suggest a relative reduction in incidence on the order of 33%. Thus, the number of infections prevented is estimated to fall between 0.7 and 1.6 per 100 needle exchange participants per year. As roughly 200 IDUs have participated in the needle exchange on a monthly basis over the time period considered, the absolute number of infections prevented is estimated to fall between 1.4 and 3.3 annually. If one assumed that 40% of all infections were transmitted sexually instead of the 30% figure used above, the estimated number of infections averted would fall between 1.2 and 2.8. Given all available information, this approach suggests that roughly speaking, the New Haven needle exchange prevented between 1 and 3 infections annually.

The second approach suggested relies on backcalculation combined with estimates of the number of drug injectors in New Haven. The backcalculation model was implemented using the Weibull incubation density estimated by Brookmeyer and Goedert (1989); the median incubation time for this function equals 10 years. Using seven years of AIDS incidence data for New Haven drug injectors yielded a mean HIV incidence rate of approximately 160 infections per year. Again assuming that 30% of all infections are sexually acquired, this leaves a baseline annual incidence rate of 112 needle borne infections among New Haven's drug injectors. Early attempts to estimate the number of drug injectors in New Haven yielded estimates in the vicinity of 2,300 (Kaplan and Soloshatz, 1993), while it is difficult to believe that more than 5% of New Haven's population of 129,000 actively inject drugs, which yields an upper bound of 6,450 for the number of active drug injectors. Considering a range on the population of IDUs from 2,000 through 7,000 suggests that the average needle exchange client population of 200 represents between 2.86% and 10% of the population of all drug injectors. Multiplying these percentages by the estimated 112 infections acquired via

Suggested Citation:"EVALUATION METHODS." Institute of Medicine and National Research Council. 1994. Proceedings--Workshop on Needle Exchange and Bleach Distribution Programs. Washington, DC: The National Academies Press. doi: 10.17226/4552.
×

needle sharing yields a baseline between 3.2 and 11.2 infections annually. Finally, applying the 33% relative reduction suggests that between 1.07 and 3.73 infections have been averted annually. If it was instead assumed that 40% of all infections derived from sexual exposure, the estimated number of infections averted would fall between 0.92 and 3.2 per year, a range very close to that obtained from the equilibrium model.

3.7 A Note on the Consequences of Laboratory Testing Errors

It seems prudent at this point to discuss briefly the consequences of laboratory testing error on the analysis above, for in effect it has been assumed that all tests on needles yield correct responses regarding HIV status. The somewhat surprising result is that were one to take laboratory testing errors into account in the analysis, one would discover that the modeled impact of needle exchange would increase.

To see why this is the case, let and denote the true average levels of infection in circulating needles before and after a needle exchange has been implemented (and assume ), and and denote the measured level of infection before and after the advent of needle exchange. Finally, let s denote the sensitivity of the PCR test (that is, the conditional probability that an HIV-positive needle will test positive), and ψ denote one minus the test specificity (so ψ is the probability that a needle erroneously tests positive given that it is in truth uncontaminated). The observed quantities are related to the true quantities by the simple equation

(25)

First, note that the apparent difference in the mean level of infection among circulating needles is given by

Suggested Citation:"EVALUATION METHODS." Institute of Medicine and National Research Council. 1994. Proceedings--Workshop on Needle Exchange and Bleach Distribution Programs. Washington, DC: The National Academies Press. doi: 10.17226/4552.
×

(26)

assuming that the probability a truly infected needle tests positive is at least as large as the probability that a truly uninfected needle tests positive, an assumption that is patently true for any reasonable HIV testing procedure. For example, in pilot trials of the PCR approach reported by Heimer et al. (1992a), 28 of 30 known HIV infected needles tested positive, while 0 of 64 known HIV negative needles tested positive. What this means is that the true reduction in the mean level of infection in circulating needles is at least as large as the measured reduction in the mean level of infection.

However, the main result of the last section relies not on the absolute reduction in the mean level of infection in circulating needles, but rather upon the relative reduction in this quantity. Here too, however, ignoring laboratory error proves to be conservative, for taking equations (25) and (26) together shows that

(27)

again providing that σψ , a perfectly reasonable assumption. Equation (27) shows that the actual relative reduction will be at least as large as the measured relative reduction in the mean level of infection in circulating needles. Thus, failing to account for laboratory testing errors actually lends the analysis a degree of conservatism not discussed until now.

3.8 A Note on Data Collection for Model Implementation Elsewhere

The syringe tracking and testing system provides an exhaustive approach to collecting the data needed for studying the physics of needle exchange. This approach proved feasible in New Haven where the volume of needles exchanged is limited due to both the number of IDUs and program policy capping the number of needles exchanged per visit. However, in other

Suggested Citation:"EVALUATION METHODS." Institute of Medicine and National Research Council. 1994. Proceedings--Workshop on Needle Exchange and Bleach Distribution Programs. Washington, DC: The National Academies Press. doi: 10.17226/4552.
×

programs serving greater numbers of clients or operating without a cap on needles exchanged per visit, it is not practical to track all distributed syringes.

What are needed are sufficient data for computing the following quantities at different points in time: the number of needles distributed per client per unit time (v); the mean needle circulation time (t), and the mean level of infection in circulating needles . If one wishes to conduct ''holdout" tests as discussed in Section 3.6, then one also needs an estimate of , that is, the mean level of infection in needles in the absence of needle exchange. Such data enable empirical tests of equations (18) and (19) which are at the heart of the theory developed.

To estimate the needle distribution rate requires knowing two things for a given time period: the total number of needles distributed, and the total number of clients served (as distinct from the total number of client visits). The first of these numbers should be easily determined from program records. If individual counting of outbound needles proves difficult because of the pressures associated with actually exchanging needles, one should still be able to determine the volume of needles distributed by subtracting the inventory of needles on hand at the end of the time period in question from the inventory of needles available at the start of the period (adjusting for the arrival of fresh inventory if necessary). The number of clients served in a given time period can be counted in a variety of ways. One approach is simply to ask clients "Is this your first visit to the program in (MONTH OF ACCOUNTING PERIOD)?" and keep track of the number of people who say yes. The required quantity n is then given by division.

Needle circulation times can be estimated via banding studies similar to that reported by Guydish et al. (1991) in San Francisco. All needles distributed on a given day can be banded with a given color or other identifying mark. When needles are returned on subsequent days, one records the number of banded needles returned each day. Doing so over time will lead to a frequency distribution of circulation time for returned needles, from which a mean can be computed. It is also important to gain a feel for the average number of needles distributed per exchange transaction, so that the mean circulation time can be normalized to a per needle measure. Short of counting all needles distributed with each transaction, it is possible to

Suggested Citation:"EVALUATION METHODS." Institute of Medicine and National Research Council. 1994. Proceedings--Workshop on Needle Exchange and Bleach Distribution Programs. Washington, DC: The National Academies Press. doi: 10.17226/4552.
×

obtain an estimate of mean batch size by sampling transactions. For example, outreach workers could record the number of needles exchanged for every fifth, or every tenth transaction, and divide the total number of needles distributed in sampled transactions by the number of sampled transactions to obtain an estimate of the mean batch size.

Finally, samples of returned needles must be tested for HIV. A simple sampling mechanism would be to maintain two sharps containers for needle return. The first one (or two) needles returned with any transaction could be deposited into the first container (the sampling container), while the remainder would be deposited into the second container. Needles selected for testing would then be selected randomly from the first container. While PCR was used to test needles in the New Haven studies, EIA tests have been used to test needles in other settings (Chitwood et al., 1990; Wodak et al., 1987), and recent modifications of EIA protocols have produced results that surpass PCR in terms of test quality (Myers et al., 1993).

4. A Roll Call of Results from the New Haven Studies: What Was Learned versus What Might Have Been

Sections 2 and 3 of this paper have presented the details of two distinct modeling approaches for use in evaluating needle exchange programs. Both of these approaches were applied to data collected in conjunction with the New Haven needle exchange, and both suggest that needle exchange has had considerable impact on HIV transmission among program participants.

What cannot be overstated, however, is the fact that had the data been different, the models could have led to different results. To be very clear about this, a roll call of the results of the New Haven studies will now be presented, along with a listing of what might have resulted from different data.

First, the change point/incidence model of Section 2 did not allow rejection of the null hypothesis that no new infections occurred among needle exchange clients. Given that more than 40% of tested needles were HIV positive, it is clear that the HIV incidence was non-zero in the past in this population. It could have been the case that the change point test led to rejecting the hypothesis of no new infection in 12 or more of the 132

Suggested Citation:"EVALUATION METHODS." Institute of Medicine and National Research Council. 1994. Proceedings--Workshop on Needle Exchange and Bleach Distribution Programs. Washington, DC: The National Academies Press. doi: 10.17226/4552.
×

instances considered. In fact, it could have been the case that many change point tests led to rejection. This would have resulted in rejection of the hypothesis of zero incidence, and led to a high estimated incidence rate. That this did not occur suggests that needle exchange is helping rather than hurting matters.

The analysis of Section 3 affords several opportunities for contemplating what might have been. It was shown that the "law of conservation of needles" held approximately. This did not have to be the case. The number of needles distributed could have exceeded the number of needles returned by 20%, or even 50%, ruining the claim that the program is a needle exchange. However, the total number of needles distributed between November 1990 and June 1993 (80,292) exceeds the total number returned to the program over the same time period (78,067) by only 2.9%.

To be effective, it was argued that needle exchange must reduce needle circulation times. That mean circulation times fell over time is clear from Figure 3. This did not have to occur. Had participating needle exchange clients failed to visit the program with increasing frequency over time, circulation times would not have fallen. It could have been the case that observed circulation times remained the same or even increased.

Equation (19) provided a direct test of the notion that increasing needle exchange frequency reduces circulation time. The data strongly supported the postulated relation. This did not have to be the case. It could have been that the relationship postulated was not borne out by the data. Equation (19) also suggests that in the absence of needle exchange, needle circulation times were in the neighborhood of three weeks! Again, it could have been the case that the estimated pre-needle exchange circulation times were no different from the circulation times observed in the study, a result that would have constituted a strike against the efficacy of needle exchange.

The mean level of infection in needles observed generally conforms to the mathematical expression derived from theory, though there are four outliers to be sure. However, it could have been the case that there was no conformance whatsoever between the theory and the data. The level of infection in needles could have remained constant over time, or could even have displayed a statistically significant upward trend, something which would have constituted a very serious strike against the needle exchange.

Suggested Citation:"EVALUATION METHODS." Institute of Medicine and National Research Council. 1994. Proceedings--Workshop on Needle Exchange and Bleach Distribution Programs. Washington, DC: The National Academies Press. doi: 10.17226/4552.
×

This did not happen. That the model was able to reasonably predict the level of infection in needles from two holdout samples, street needles and underground exchange needles, provides additional confidence in the results.

Of the two modeling approaches advanced, the methods of Section 2 yield a maximum likelihood incidence estimate of zero, while the methods of Section 3 produce a range of post-needle exchange incidence estimates from one through four per hundred IDUs per year. These two results are not inconsistent, for while the best incidence point estimate from Section 2 is zero, rates as high as four per hundred per year are well within the noise level associated with the estimation technique.

In sum, the models advocated could have led to conclusions suggesting that needle exchange is ineffective. That they have led to the opposite conclusion, namely, that needle exchange has prevented infections among program clients, is a result of the data, not a result of the models. The modeling approaches advocated thus meet what is perhaps the most basic premise of all scientific research, namely, that of falsifiability. Things could have gone the other way. They did not.

5. A Note on Validating Mathematical Models with Special Reference to AIDS

How should one think about the validation of mathematical models in general? While being able to fit a model to observed data is clearly an important criterion, it is not the only one. Norman Bailey (1994) identifies other criteria to consider when attempting to validate public health models of HIV and AIDS, including cross-checking with other research results and preliminary prediction of future events (or backcasting of prior events). That the majority of needle exchange research published to date claims to support the efficacy of needle exchange cannot be disputed. Earlier modeling studies relying on behavioral data and assumptions in addition to only limited syringe tracking and testing data also led to similar conclusions (Kaplan and O'Keefe, 1993). In the case of the "holdout" samples described earlier, it was possible to successfully backcast the level of infection in needles for data that had not been employed in calibrating the model.

Suggested Citation:"EVALUATION METHODS." Institute of Medicine and National Research Council. 1994. Proceedings--Workshop on Needle Exchange and Bleach Distribution Programs. Washington, DC: The National Academies Press. doi: 10.17226/4552.
×

That models are routinely employed in studies of the AIDS epidemic is perhaps a theme worth pursuing momentarily. Consider the simple problem of estimating the number of HIV infections that have occurred in the United States. Many readers will be familiar with estimates in the neighborhood of 1 million infected persons, but how does one know? No one has ever counted! The most credible estimates are those that derive from a mathematical model, specifically, the backcalculation method as exemplified by the work of Brookmeyer (1991) and Brookmeyer and Gail (1988). Why should one believe the resulting estimates from backcalculation? It is possible to match predicted AIDS incidence with what is observed, but HIV incidence remains invisible! The faith in backcalculation follows to a great extent from one's belief in the process represented by the model.

A similar example involves studies that have attempted to estimate the probability of HIV transmission during individual acts of sexual intercourse between discordant pairs (i.e. one infected, one uninfected). These studies have not been conducted in a manner where the uninfected partner is tested following each act of unprotected sex for obvious reasons. The most commonly cited estimates of the per sex act infectivity follow from very simple models of risk compounding and population heterogeneity (Kaplan, 1990; Shiboski and Jewell, 1992; Wiley et al., 1989). Again, it is the process embodied in the modeling which leads one to gain some confidence in the nature of the results obtained.

What modeling offers is a language for making these processes explicit. In fact, modeling offers something that a controlled experiment cannot. Even if one discovered via a controlled experiment that the HIV transmission rate among needle exchange participants was less than among an otherwise equivalent group of IDUs without access to needle exchange, one would not know why needle exchange had the effect it had. One would not be able to formulate suggestions for what to expect from needle exchanges operating under a different set of circumstances from those in the experiment. To be sure, a controlled experiment would offer the best evidence possible for the effect of the specific needle exchange program tested. Precise measurements for one such program, however, may not be worth as much as a set of general principles that can be explored in a variety of settings (such as the

Suggested Citation:"EVALUATION METHODS." Institute of Medicine and National Research Council. 1994. Proceedings--Workshop on Needle Exchange and Bleach Distribution Programs. Washington, DC: The National Academies Press. doi: 10.17226/4552.
×

relationships between needle exchange rates, needle circulation times, and the mean level of infection in circulating needles, for example). The generation of such principles is one of the major benefits of modeling.

6. Concluding Thoughts

Operational modeling is not the method of choice most needle exchange researchers have settled upon, and it is not the intent of this paper to suggest that the techniques developed here are the only ones worth pursuing. It is, however, important to stress the unique features of this approach to program evaluation. The operational modeling of needle exchange leads to a number of falsifiable conjectures regarding program function and HIV transmission risks that can be tested empirically. Operational modeling relies on objectively observed program data and other available public health information as opposed to behavioral information estimated from surveys of needle exchange clients. The modeling philosophy espoused throughout this paper is but an instance of the application of the methods of operations research which have been applied successfully to business (Cook and Russell, 1989), industry (Nahmias, 1989), public service systems (Drake et al., 1972; Larson and Odoni, 1981) and health care (Eddy, 1980; Willemain and Larson, 1977). When viewed as a methodology for evaluating an AIDS prevention program such as needle exchange, operational modeling may appear both suspicious and arcane. However, when viewed within the historical context of the field of operations research, the approach espoused is well-grounded methodologically.

The purpose of evaluating programs in general and needle exchange in particular is to provide information useful for decision making (Larson and Kaplan, 1981). Public health decision makers seriously interested in needle exchange do not need information regarding program efficacy with the same precision required to land a spacecraft on the moon. Rather, the task at hand is to decide if sufficient evidence exists to break the state of equipoise regarding the usefulness of needle exchange. The results of the New Haven studies add weight to the arguments of those who advocate needle

Suggested Citation:"EVALUATION METHODS." Institute of Medicine and National Research Council. 1994. Proceedings--Workshop on Needle Exchange and Bleach Distribution Programs. Washington, DC: The National Academies Press. doi: 10.17226/4552.
×

exchange programs on the grounds of HIV incidence reduction. Whether this additional weight is sufficient to break the state of equipoise, however, is for public health policy officials to decide.

To recap, this paper has presented the details of a new set of techniques developed explicitly for the purpose of evaluating needle exchange programs. These methods have been applied to the evaluation of the legal needle exchange program operated by the New Haven Health Department. Field data filtered through the lens of operational modeling suggest strongly that the operations of needle exchange have reduced needle-borne HIV transmission risks among participating drug injectors. The weight of the evidence greatly favors the contention that, at least in New Haven, HIV transmission has slowed as a result of needle exchange.

Acknowledgments

The PCR approach to testing needles for the presence of HIV proviral DNA was developed by Dr. Robert Heimer, Assistant Professor of Medicine at Yale University. He has supervised all laboratory operations from the start of our evaluation studies with the able assistance of Ms. Bini Jariwala. The syringe tracking and testing database has grown considerably, and managing this database is the responsibility of Ms. Coleen Klansky at the Yale University School of Management. It is frightening to imagine how this project would fare without her skills in information management. Mr. Kaveh Khoshnood, a doctoral candidate in Epidemiology and Public Health, continues to provide research assistance in the evaluation of the New Haven needle exchange. This research does not take place in a vacuum, and I feel it appropriate to acknowledge the stimulating environment provided by other Yale colleagues dedicated to AIDS research of all stripes, including Drs. Phil Alcabes, Rick Altice, Gerald Friedland, Jeanette Ickovics, Alvin Novick, Peter Selwyn, Ann Williams, and David Paltiel (now at Harvard's School of Public Health). I thank Dr. Edwin Cadman, Chair of the Department of Internal Medicine at Yale, for having sufficient interest in this research to pay for it when no external funding was available. That situation has thankfully changed, and it is a pleasure to acknowledge the National Institute on Drug Abuse for support under grant R01-DA07676-02 (Dr.

Suggested Citation:"EVALUATION METHODS." Institute of Medicine and National Research Council. 1994. Proceedings--Workshop on Needle Exchange and Bleach Distribution Programs. Washington, DC: The National Academies Press. doi: 10.17226/4552.
×

Peter Hartsock, Project Officer) and the Robert Wood Johnson Foundation for support under grant 20049 (Dr. Paul Jellinek, Project Officer). Finally, none of this work would have been possible without the cooperation of the New Haven Health Department. Regardless of one's feelings about needle exchange, no one can deny that Ms. Elaine O'Keefe, former AIDS Division Director; Mr. William Quinn, Director of Health Services; and the entire outreach staff of the needle exchange program care deeply for the clients they serve. Their concern and dedication cannot be overstated.

References

Aldrich MR, Mandel J and Newmeyer JA (1990). A spreadsheet for AIDS: Estimating heterosexual injection drug user population size from AIDS statistics in San Francisco. Journal of Psychoactive Drugs 22(3):343-349.

Anderson RM and May RM (1991). Infectious diseases of humans: Dynamics and control. Oxford University Press, Oxford, England.


Bacchetti P and Moss AR (1989). Incubation period of AIDS in San Francisco. Nature 338:251-253.

Bailey NJT (1994). Prediction and validation in the public health modelling of HIV/AIDS. Statistics in Medicine (in press).

Bhat UN (1984). Elements of applied stochastic processes. John Wiley and Sons, New York.

Brookmeyer R (1991). Reconstruction and future trends of the AIDS epidemic in the United States. Science 253:37-42.

Brookmeyer R and Gail M (1988). A method for obtaining short-term projections and lower bounds on the size of the AIDS epidemic. Journal of the American Statistical Association 83:301-308.

Brookmeyer R and Goedert JJ (1989). Censoring in an epidemic with application to hemophilia associated AIDS. Biometrics 45:325-335.


CDC (1990). National HIV seroprevalence surveys. Centers for Disease Control and Prevention, Atlanta (HIV/CID/9-90/006, Table 1).

Chitwood D, McCoy C, Inciardi J, et al. (1990). HIV seropositivity of needles from shooting galleries in south Florida. American Journal of Public Health 80:150-152.

Cook TM and Russell RA (1989). Introduction to management science. Prentice Hall, Englewood Cliffs, New Jersey.

Suggested Citation:"EVALUATION METHODS." Institute of Medicine and National Research Council. 1994. Proceedings--Workshop on Needle Exchange and Bleach Distribution Programs. Washington, DC: The National Academies Press. doi: 10.17226/4552.
×

Cox DR (1962). Renewal theory. Methuen and Company, London, England.

Cox DR (1970). The analysis of binary data. Chapman and Hall, London, England.

Cox DR and Hinkley DV (1974). Theoretical statistics. Chapman and Hall, London, England.

Des Jarlais DC and Friedman SR (1992). AIDS and legal access to sterile drug injection equipment. Annals of the American Academy of Political and Social Science 521:42-65.

Donoghoe M, Stimson G, Dolan K and Alldritt L. (1989). Changes in HIV risk behaviour in clients of syringe exchange schemes in England and Scotland. AIDS 3:267-272.

Drake AW (1967). Fundamentals of applied probability theory. McGraw Hill, New York.

Drake AW, Keeney RL and Morse PM (eds.) (1972). Analysis of public systems. MIT Press, Cambridge, Massachusetts.


Eddy DM (1980). Screening for cancer: Theory, analysis, and design. Prentice-Hall, Englewood Cliffs, New Jersey.


Freund HP and Book DL (1990). Determination of the spread of HIV from the AIDS incidence history. Mathematical Biosciences 98:227-241.

Frischer M, Leyland A, Cormack R et al. (1993). Estimating the population prevalence of injection drug use and infection with human immunodeficiency virus among injection drug users in Glasgow, Scotland. American Journal of Epidemiology 138:170-181.


GAO(1993). Needle exchange programs: Research suggests promise as an AIDS prevention strategy. United States General Accounting Office Report Number GAO/HRD-93-60, Washington, DC.

GAUSS (1992). Maximum likelihood. GAUSS 3.0 Applications, The GAUSS System Version 3.0, Aptech Systems Incorporated, Maple Valley, Washington.

Gross D and Harris CM (1985). Fundamentals of queuing theory. John Wiley and Sons, New York.

Guydish J, Clark G, Garcia D et al. (1991). Evaluating needle exchange: Do distributed needles come back? American Journal of Public Health 81:617619.


Hagan H, Des Jarlais DC, Purchase D et al. (1991). The Tacoma syringe exchange. Journal of Addictive Diseases 10:81-88.

Hartgers C, Buning E, van Santen G et al. (1989). The impact of the needle and syringe-exchange programme in Amsterdam on injecting risk behaviour. AIDS 3:571-576.

Suggested Citation:"EVALUATION METHODS." Institute of Medicine and National Research Council. 1994. Proceedings--Workshop on Needle Exchange and Bleach Distribution Programs. Washington, DC: The National Academies Press. doi: 10.17226/4552.
×

Heimer R, Myers S, Cadman E et al. (1992a). Detection by polymerase chain reaction of human immunodeficiency virus Type I proviral DNA sequences in needles of injecting drug users. Journal of Infectious Diseases 165:781782.

Heimer R, Kaplan EH and Cadman C (1992b). Prevalence of HIV-infected syringes during a syringe exchange program. New England Journal of Medicine 327:1883-1884.

Heimer R, Kaplan EH, Khoshnood K et al. (1993). Needle exchange decreases the prevalence of HIV-1 proviral DNA in returned syringes in New Haven, Connecticut. American Journal of Medicine 95:214-220.

Joseph SC and Des Jarlais DC (1989). Needle and syringe exchange as a method of AIDS epidemic control. AIDS Updates 2:1-8.


Kaplan EH (1989). What are the risks of risky sex? Modeling the AIDS epidemic. Operations Research 37:198-209.

Kaplan EH (1990). Modeling HIV infectivity: Must sex acts be counted? Journal of Acquired Immune Deficiency Syndromes 3:55-61.

Kaplan EH (1994). A method for evaluating needle exchange programs. Statistics in Medicine (in press).

Kaplan EH and Heimer R (1992). HIV prevalence among intravenous drug users: Model-based estimates from New Haven's legal needle exchange. Journal of Acquired Immune Deficiency Syndromes 5:163-169.

Kaplan EH and Heimer R (1993). What happened to HIV transmission among drug injectors in New Haven? Chance 6(2):9-14.

Kaplan EH and Heimer R (1994). HIV incidence among needle exchange participants: Estimates from syringe tracking and testing data. Journal of Acquired Immune Deficiency Syndromes 7:182-189.

Kaplan EH and O'Keefe E. (1993). Let the needles do the talking! Evaluating the New Haven needle exchange. Interfaces 23(1):7-26.

Kaplan EH and Soloshatz D (1993). How many drug injectors are there in New Haven? Answers from AIDS data. Mathematical and Computer Modelling 17:109-115.


Larson RC and Kaplan EH (1981). Decision-oriented approaches to program evaluation. New Directions for Program Evaluation 10:49-68.

Larson RC and Odoni AR (1981). Urban Operations Research. Prentice-Hall, Englewood Cliffs, New Jersey.

Ljungberg B, Christensson B, Tunving K et al. (1991). HIV prevention among injecting drug users: Three years of experience from a syringe exchange program in Sweden. Journal of Acquired Immune Deficiency Syndromes 4:890-895.

Suggested Citation:"EVALUATION METHODS." Institute of Medicine and National Research Council. 1994. Proceedings--Workshop on Needle Exchange and Bleach Distribution Programs. Washington, DC: The National Academies Press. doi: 10.17226/4552.
×

Mausner JS and Kramer S (1985). Epidemiology: An introductory text. W.B. Saunders Company, Philadelphia, Pennsylvania.

Muñoz A, Wang M-C, Bass S et al. (1989). Acquired immunodeficiency syndrome (AIDS)-free time after human immunodeficiency virus type 1 (HIV-1) seroconversion in homosexual men. American Journal of Epidemiology 130:530539.

Myers SS, Heimer R, Liu D and Henrard D (1993). HIV DNA and antibodies in syringes from injecting drug users: A comparison of detection techniques. AIDS 7:925-931.


Nahmias S (1989). Production and operations analysis. Richard D. Irwin Incorporated, Boston, Massachusetts.

Nelson KE, Vlahov D, Muñoz A et al. (1992). Incident HIV-1 infections in a cohort of intravenous drug users (IVDUs). Presented at the VIII International Conference on AIDS, Amsterdam, The Netherlands (Abstract PoC 4698).

Newmeyer JA (1988). The prevalence of drug use in San Francisco in 1987. Journal of Psychoactive Drugs 20:185-189.


Page ES (1955). A test for a change in a parameter occurring at an unknown point. Biometrika 42:523-527.


Schoenbaum EE, Hartel D, Selwyn PA et al. (1989). Risk factors for human immunodeficiency virus infection in intravenous drug users. New England Journal of Medicine 321:874-879.

Shiboski SC and Jewell NP (1992). Statistical analysis of the time dependence of HIV infectivity based on partner study data. Journal of the American Statistical Association 87:360-372.

Stimson G (1989). Syringe exchange programmes for injecting drug users. AIDS 3:253-260.


Valleroy L, Weinstein B, Groseclose S et al. (1993). Evaluating the impact of a new needle/syringe law: Surveillance of needle/syringe sales at Connecticut pharmacies. Presented at the IXth International Conference on AIDS, Berlin, Germany (Abstract PO-C24-3189).


Watters J, Cheng Y-T and the Prevention Point Research Group (1991). Syringe exchange (SE) in San Francisco: Preliminary findings. Presented at the VII International Conference on AIDS, Florence, Italy (Abstract ThC 99).

Weinstein B and Hadler JL (1993). Impact of new legislation on needle and syringe purchase and possession-Connecticut, 1992. Morbidity and Mortality Weekly Report 42:145-148.

Wiley JA, Herschkorn SJ and Padian NS (1989). Heterogeneity in the probability of HIV transmission per sexual contact: The case of male-to-female transmission in penile-vaginal intercourse. Statistics in Medicine 8:93-102.

Suggested Citation:"EVALUATION METHODS." Institute of Medicine and National Research Council. 1994. Proceedings--Workshop on Needle Exchange and Bleach Distribution Programs. Washington, DC: The National Academies Press. doi: 10.17226/4552.
×

Willemain TR and Larson RC (1977). Emergency medical systems analysis: Papers on the planning and evaluation of services. Lexington Books, D.C. Heath and Company, Lexington, Massachusetts.

Wodak A, Dolan K, Imrie A et al. (1987). Antibodies to human immunodeficiency virus in needles and syringes used by intravenous drug abusers. Medical Journal of Australia 147:275-276.

Suggested Citation:"EVALUATION METHODS." Institute of Medicine and National Research Council. 1994. Proceedings--Workshop on Needle Exchange and Bleach Distribution Programs. Washington, DC: The National Academies Press. doi: 10.17226/4552.
×

Figure 1

Client Participation and Visitation

Suggested Citation:"EVALUATION METHODS." Institute of Medicine and National Research Council. 1994. Proceedings--Workshop on Needle Exchange and Bleach Distribution Programs. Washington, DC: The National Academies Press. doi: 10.17226/4552.
×

Figure 2

Volume of Needle Exchange

Suggested Citation:"EVALUATION METHODS." Institute of Medicine and National Research Council. 1994. Proceedings--Workshop on Needle Exchange and Bleach Distribution Programs. Washington, DC: The National Academies Press. doi: 10.17226/4552.
×

Figure 3

Mean Needle Circulation Time

Suggested Citation:"EVALUATION METHODS." Institute of Medicine and National Research Council. 1994. Proceedings--Workshop on Needle Exchange and Bleach Distribution Programs. Washington, DC: The National Academies Press. doi: 10.17226/4552.
×

Figure 4

Needle Exchange and Removal Rates

Suggested Citation:"EVALUATION METHODS." Institute of Medicine and National Research Council. 1994. Proceedings--Workshop on Needle Exchange and Bleach Distribution Programs. Washington, DC: The National Academies Press. doi: 10.17226/4552.
×

Figure 5

HIV Infection in Needles over Time

Suggested Citation:"EVALUATION METHODS." Institute of Medicine and National Research Council. 1994. Proceedings--Workshop on Needle Exchange and Bleach Distribution Programs. Washington, DC: The National Academies Press. doi: 10.17226/4552.
×

DISCUSSION: EVALUATION METHODS

ROBERT BOOTH

Robert Booth observed that, on one hand, quantitative evaluation methods are especially appealing because of the typically large sample sizes they employ, their use of statistical inference, and their generalizability. On the other hand, they are subject to biases and problems of recall and social desirability. Such methods are also subject to threats to internal validity, particularly one-shot case studies and studies using cross-sectional designs, in which it is difficult to parcel out the influence of historical trends on observed differences. Some statistical procedures have been developed to attend to these difficulties, but the use of experimental designs that strengthen the grounds for inference is preferable.

He commented that qualitative methods are also appealing, in that the yield from qualitative methods has a richness and detail often lacking in quantitative approaches. In addition, qualitative methods can uncover information that survey research often misses. However, qualitative research is often threatened by issues of sampling bias and sample size.

Booth described mathematical modeling as a particularly strong methodology, especially as operationalized by Edward Kaplan. Kaplan's approach is intuitively appealing; it yields a conservative estimate of effectiveness; its limitations, Booth noted, are its exclusive focus on needles and their exchange and the attribution of HIV reduction solely to needle exchange. Other interventions and other factors may also play a role in how injectors obtain and use needles. In addition, although exposure time can be calculated in the model, because the focus is on needles and not on people, one does not know how many people use a needle; one knows only the length of time the needle was in circulation.

Looking at the context of program evaluation, particularly quantitative methods, which have dominated the literature to date, Booth stated that the evidence clearly suggests that HIV risk behaviors, particularly needle risk behaviors, have declined over time. The decline is possibly due to the interventions that have been implemented, the media, word of mouth, or other factors. Evidence of the decline, however, has been found in cross-sectional research designs, one-shot case studies, pre-post-quasi-experimental designs, true experimental designs, time series designs, and other designs.

He said that evidence also suggests that risk has not been totally eliminated; addicts sometimes find themselves in situations in which it is difficult to follow risk reduction protocols. These findings are supported in evaluations of drug treatment programs, community-based outreach interventions, educational office-based interventions, pre-and post-HIV testing and counseling, and needle exchanges. Indeed, risk reduction in the absence of intervention has also been reported.

This is not to say, he went on, that any and all interventions work or that behaviors will change without interventions, although they may. Nor does it mean that needle exchanges are no better or worse than other interventions. Indeed, data have

Suggested Citation:"EVALUATION METHODS." Institute of Medicine and National Research Council. 1994. Proceedings--Workshop on Needle Exchange and Bleach Distribution Programs. Washington, DC: The National Academies Press. doi: 10.17226/4552.
×

shown that participants in needle exchange programs differ demographically from participants in drug treatment and other interventions and that participants in drug treatment differ from those involved in street-based outreach programs. As was said a number of times during the workshop, different programs attract different types of people and people at different stages of their addiction, and all the intervention programs may be necessary.

What are lacking in the evaluations discussed over the past two days, other than in a few studies, are rigorous tests of program effectiveness, Booth concluded. Nearly 10 years ago, Mark Lipsey wrote an article entitled ''Evaluation: The State of the Art and the Sorry State of the Science" (Lipsey, Crosse, Dunkle, et al., 1985). He noted that most studies conducted under the label of program evaluation embody at least the rudiments of experimental methods. The telltale signs of quantitative measurement are there, some attempts have usually been made at control comparison or use of baseline groups, and cause-and-effect thinking is evident, even when there is little resemblance to classical randomized experiments. However, he said, program evaluation is often poorly done within the experimental paradigms, and he cited two reasons for this. First, there are numerous practical difficulties in matching good research designs to practical program circumstances. Second, social scientists, for the most part, are not very well trained to do methodologically exacting research under field conditions. To this Booth added a third reason: outcome evaluation is more exciting than process evaluation. Hence, in his view, program evaluation has not only neglected theory, but attention to the treatments or interventions provided—that is, the intervention dose—has been neglected. The result is that treatments or interventions have been represented as black boxes. Moreover, the vast majority of studies of risk reduction programs that involve some sort of experimental design represent the treatment or intervention level as a dichotomy. That is, they assume that each member of the treatment or intervention group receives the same treatment and each member of the control group receives nothing. When outcomes show success, they are due to the treatment, and when outcomes show failure, they are due to failure of the treatment. However, the lack of outcome success may be due to the failure of the research, not of the treatment or intervention.

Needle exchange programs are particularly difficult to evaluate, he observed, in part because the standards for good research are more stringent than for other interventions, notably community-based outreach interventions, and in part because the research itself may be a deterrent to program participation. Consequently, Booth recommended the use of multiple methods to evaluate needle exchange programs, including quantitative and qualitative studies, mathematical modeling, and use of what have been referred to as focal-local indicators, such as those used in the Tacoma and Portland studies reported on earlier, in which the researchers looked at hepatitis-B prevalence and incidence over time. In addition to hepatitis data, other indicators would include HIV and AIDS cases, drug-related crimes and arrests, and sexually transmitted diseases. Such investigations would require an extended period of time. In addition, he asked for recognition that any single indicator, as well as any single method, has its limitations.

Suggested Citation:"EVALUATION METHODS." Institute of Medicine and National Research Council. 1994. Proceedings--Workshop on Needle Exchange and Bleach Distribution Programs. Washington, DC: The National Academies Press. doi: 10.17226/4552.
×

He summarized by saying that most effectiveness studies have not taken into account the influence of other interventions or temporal trends in subject populations. Participants in needle exchanges or community outreach programs are considered as existing in a vacuum. Sterile needles can be obtained from sources other than needle exchanges, and behavior modifications can occur, in the absence or in the presence of needle exchange, unrelated to needle exchange. Noting that a number of presenters at the workshop have argued for looking at multiple interventions rather than attribute change to single interventions, he concluded that there is a need to focus on single interventions while measuring the landscape of other possible influential factors.

Booth ended by mentioning the important issue of retention, which has been stressed repeatedly in evaluations of drug treatment programs. In the context of needle exchange programs, the issue becomes: What are the criteria for success regarding retention in a program?

REFERENCE

Lipsey, M. W., S. Crosse, J. Dunkle, J. Pollard, and G. Stobart 1985 Evaluation: The state of the art and the sorry state of the science. New Directions for Testing and Measurement 27:7-28.

Suggested Citation:"EVALUATION METHODS." Institute of Medicine and National Research Council. 1994. Proceedings--Workshop on Needle Exchange and Bleach Distribution Programs. Washington, DC: The National Academies Press. doi: 10.17226/4552.
×
Page 153
Suggested Citation:"EVALUATION METHODS." Institute of Medicine and National Research Council. 1994. Proceedings--Workshop on Needle Exchange and Bleach Distribution Programs. Washington, DC: The National Academies Press. doi: 10.17226/4552.
×
Page 154
Suggested Citation:"EVALUATION METHODS." Institute of Medicine and National Research Council. 1994. Proceedings--Workshop on Needle Exchange and Bleach Distribution Programs. Washington, DC: The National Academies Press. doi: 10.17226/4552.
×
Page 155
Suggested Citation:"EVALUATION METHODS." Institute of Medicine and National Research Council. 1994. Proceedings--Workshop on Needle Exchange and Bleach Distribution Programs. Washington, DC: The National Academies Press. doi: 10.17226/4552.
×
Page 156
Suggested Citation:"EVALUATION METHODS." Institute of Medicine and National Research Council. 1994. Proceedings--Workshop on Needle Exchange and Bleach Distribution Programs. Washington, DC: The National Academies Press. doi: 10.17226/4552.
×
Page 157
Suggested Citation:"EVALUATION METHODS." Institute of Medicine and National Research Council. 1994. Proceedings--Workshop on Needle Exchange and Bleach Distribution Programs. Washington, DC: The National Academies Press. doi: 10.17226/4552.
×
Page 158
Suggested Citation:"EVALUATION METHODS." Institute of Medicine and National Research Council. 1994. Proceedings--Workshop on Needle Exchange and Bleach Distribution Programs. Washington, DC: The National Academies Press. doi: 10.17226/4552.
×
Page 159
Suggested Citation:"EVALUATION METHODS." Institute of Medicine and National Research Council. 1994. Proceedings--Workshop on Needle Exchange and Bleach Distribution Programs. Washington, DC: The National Academies Press. doi: 10.17226/4552.
×
Page 160
Suggested Citation:"EVALUATION METHODS." Institute of Medicine and National Research Council. 1994. Proceedings--Workshop on Needle Exchange and Bleach Distribution Programs. Washington, DC: The National Academies Press. doi: 10.17226/4552.
×
Page 161
Suggested Citation:"EVALUATION METHODS." Institute of Medicine and National Research Council. 1994. Proceedings--Workshop on Needle Exchange and Bleach Distribution Programs. Washington, DC: The National Academies Press. doi: 10.17226/4552.
×
Page 162
Suggested Citation:"EVALUATION METHODS." Institute of Medicine and National Research Council. 1994. Proceedings--Workshop on Needle Exchange and Bleach Distribution Programs. Washington, DC: The National Academies Press. doi: 10.17226/4552.
×
Page 163
Suggested Citation:"EVALUATION METHODS." Institute of Medicine and National Research Council. 1994. Proceedings--Workshop on Needle Exchange and Bleach Distribution Programs. Washington, DC: The National Academies Press. doi: 10.17226/4552.
×
Page 164
Suggested Citation:"EVALUATION METHODS." Institute of Medicine and National Research Council. 1994. Proceedings--Workshop on Needle Exchange and Bleach Distribution Programs. Washington, DC: The National Academies Press. doi: 10.17226/4552.
×
Page 165
Suggested Citation:"EVALUATION METHODS." Institute of Medicine and National Research Council. 1994. Proceedings--Workshop on Needle Exchange and Bleach Distribution Programs. Washington, DC: The National Academies Press. doi: 10.17226/4552.
×
Page 166
Suggested Citation:"EVALUATION METHODS." Institute of Medicine and National Research Council. 1994. Proceedings--Workshop on Needle Exchange and Bleach Distribution Programs. Washington, DC: The National Academies Press. doi: 10.17226/4552.
×
Page 167
Suggested Citation:"EVALUATION METHODS." Institute of Medicine and National Research Council. 1994. Proceedings--Workshop on Needle Exchange and Bleach Distribution Programs. Washington, DC: The National Academies Press. doi: 10.17226/4552.
×
Page 168
Suggested Citation:"EVALUATION METHODS." Institute of Medicine and National Research Council. 1994. Proceedings--Workshop on Needle Exchange and Bleach Distribution Programs. Washington, DC: The National Academies Press. doi: 10.17226/4552.
×
Page 169
Suggested Citation:"EVALUATION METHODS." Institute of Medicine and National Research Council. 1994. Proceedings--Workshop on Needle Exchange and Bleach Distribution Programs. Washington, DC: The National Academies Press. doi: 10.17226/4552.
×
Page 170
Suggested Citation:"EVALUATION METHODS." Institute of Medicine and National Research Council. 1994. Proceedings--Workshop on Needle Exchange and Bleach Distribution Programs. Washington, DC: The National Academies Press. doi: 10.17226/4552.
×
Page 171
Suggested Citation:"EVALUATION METHODS." Institute of Medicine and National Research Council. 1994. Proceedings--Workshop on Needle Exchange and Bleach Distribution Programs. Washington, DC: The National Academies Press. doi: 10.17226/4552.
×
Page 172
Suggested Citation:"EVALUATION METHODS." Institute of Medicine and National Research Council. 1994. Proceedings--Workshop on Needle Exchange and Bleach Distribution Programs. Washington, DC: The National Academies Press. doi: 10.17226/4552.
×
Page 173
Suggested Citation:"EVALUATION METHODS." Institute of Medicine and National Research Council. 1994. Proceedings--Workshop on Needle Exchange and Bleach Distribution Programs. Washington, DC: The National Academies Press. doi: 10.17226/4552.
×
Page 174
Suggested Citation:"EVALUATION METHODS." Institute of Medicine and National Research Council. 1994. Proceedings--Workshop on Needle Exchange and Bleach Distribution Programs. Washington, DC: The National Academies Press. doi: 10.17226/4552.
×
Page 175
Suggested Citation:"EVALUATION METHODS." Institute of Medicine and National Research Council. 1994. Proceedings--Workshop on Needle Exchange and Bleach Distribution Programs. Washington, DC: The National Academies Press. doi: 10.17226/4552.
×
Page 176
Suggested Citation:"EVALUATION METHODS." Institute of Medicine and National Research Council. 1994. Proceedings--Workshop on Needle Exchange and Bleach Distribution Programs. Washington, DC: The National Academies Press. doi: 10.17226/4552.
×
Page 177
Suggested Citation:"EVALUATION METHODS." Institute of Medicine and National Research Council. 1994. Proceedings--Workshop on Needle Exchange and Bleach Distribution Programs. Washington, DC: The National Academies Press. doi: 10.17226/4552.
×
Page 178
Suggested Citation:"EVALUATION METHODS." Institute of Medicine and National Research Council. 1994. Proceedings--Workshop on Needle Exchange and Bleach Distribution Programs. Washington, DC: The National Academies Press. doi: 10.17226/4552.
×
Page 179
Suggested Citation:"EVALUATION METHODS." Institute of Medicine and National Research Council. 1994. Proceedings--Workshop on Needle Exchange and Bleach Distribution Programs. Washington, DC: The National Academies Press. doi: 10.17226/4552.
×
Page 180
Suggested Citation:"EVALUATION METHODS." Institute of Medicine and National Research Council. 1994. Proceedings--Workshop on Needle Exchange and Bleach Distribution Programs. Washington, DC: The National Academies Press. doi: 10.17226/4552.
×
Page 181
Suggested Citation:"EVALUATION METHODS." Institute of Medicine and National Research Council. 1994. Proceedings--Workshop on Needle Exchange and Bleach Distribution Programs. Washington, DC: The National Academies Press. doi: 10.17226/4552.
×
Page 182
Suggested Citation:"EVALUATION METHODS." Institute of Medicine and National Research Council. 1994. Proceedings--Workshop on Needle Exchange and Bleach Distribution Programs. Washington, DC: The National Academies Press. doi: 10.17226/4552.
×
Page 183
Suggested Citation:"EVALUATION METHODS." Institute of Medicine and National Research Council. 1994. Proceedings--Workshop on Needle Exchange and Bleach Distribution Programs. Washington, DC: The National Academies Press. doi: 10.17226/4552.
×
Page 184
Suggested Citation:"EVALUATION METHODS." Institute of Medicine and National Research Council. 1994. Proceedings--Workshop on Needle Exchange and Bleach Distribution Programs. Washington, DC: The National Academies Press. doi: 10.17226/4552.
×
Page 185
Suggested Citation:"EVALUATION METHODS." Institute of Medicine and National Research Council. 1994. Proceedings--Workshop on Needle Exchange and Bleach Distribution Programs. Washington, DC: The National Academies Press. doi: 10.17226/4552.
×
Page 186
Suggested Citation:"EVALUATION METHODS." Institute of Medicine and National Research Council. 1994. Proceedings--Workshop on Needle Exchange and Bleach Distribution Programs. Washington, DC: The National Academies Press. doi: 10.17226/4552.
×
Page 187
Suggested Citation:"EVALUATION METHODS." Institute of Medicine and National Research Council. 1994. Proceedings--Workshop on Needle Exchange and Bleach Distribution Programs. Washington, DC: The National Academies Press. doi: 10.17226/4552.
×
Page 188
Suggested Citation:"EVALUATION METHODS." Institute of Medicine and National Research Council. 1994. Proceedings--Workshop on Needle Exchange and Bleach Distribution Programs. Washington, DC: The National Academies Press. doi: 10.17226/4552.
×
Page 189
Suggested Citation:"EVALUATION METHODS." Institute of Medicine and National Research Council. 1994. Proceedings--Workshop on Needle Exchange and Bleach Distribution Programs. Washington, DC: The National Academies Press. doi: 10.17226/4552.
×
Page 190
Suggested Citation:"EVALUATION METHODS." Institute of Medicine and National Research Council. 1994. Proceedings--Workshop on Needle Exchange and Bleach Distribution Programs. Washington, DC: The National Academies Press. doi: 10.17226/4552.
×
Page 191
Suggested Citation:"EVALUATION METHODS." Institute of Medicine and National Research Council. 1994. Proceedings--Workshop on Needle Exchange and Bleach Distribution Programs. Washington, DC: The National Academies Press. doi: 10.17226/4552.
×
Page 192
Suggested Citation:"EVALUATION METHODS." Institute of Medicine and National Research Council. 1994. Proceedings--Workshop on Needle Exchange and Bleach Distribution Programs. Washington, DC: The National Academies Press. doi: 10.17226/4552.
×
Page 193
Suggested Citation:"EVALUATION METHODS." Institute of Medicine and National Research Council. 1994. Proceedings--Workshop on Needle Exchange and Bleach Distribution Programs. Washington, DC: The National Academies Press. doi: 10.17226/4552.
×
Page 194
Suggested Citation:"EVALUATION METHODS." Institute of Medicine and National Research Council. 1994. Proceedings--Workshop on Needle Exchange and Bleach Distribution Programs. Washington, DC: The National Academies Press. doi: 10.17226/4552.
×
Page 195
Suggested Citation:"EVALUATION METHODS." Institute of Medicine and National Research Council. 1994. Proceedings--Workshop on Needle Exchange and Bleach Distribution Programs. Washington, DC: The National Academies Press. doi: 10.17226/4552.
×
Page 196
Suggested Citation:"EVALUATION METHODS." Institute of Medicine and National Research Council. 1994. Proceedings--Workshop on Needle Exchange and Bleach Distribution Programs. Washington, DC: The National Academies Press. doi: 10.17226/4552.
×
Page 197
Suggested Citation:"EVALUATION METHODS." Institute of Medicine and National Research Council. 1994. Proceedings--Workshop on Needle Exchange and Bleach Distribution Programs. Washington, DC: The National Academies Press. doi: 10.17226/4552.
×
Page 198
Suggested Citation:"EVALUATION METHODS." Institute of Medicine and National Research Council. 1994. Proceedings--Workshop on Needle Exchange and Bleach Distribution Programs. Washington, DC: The National Academies Press. doi: 10.17226/4552.
×
Page 199
Suggested Citation:"EVALUATION METHODS." Institute of Medicine and National Research Council. 1994. Proceedings--Workshop on Needle Exchange and Bleach Distribution Programs. Washington, DC: The National Academies Press. doi: 10.17226/4552.
×
Page 200
Suggested Citation:"EVALUATION METHODS." Institute of Medicine and National Research Council. 1994. Proceedings--Workshop on Needle Exchange and Bleach Distribution Programs. Washington, DC: The National Academies Press. doi: 10.17226/4552.
×
Page 201
Suggested Citation:"EVALUATION METHODS." Institute of Medicine and National Research Council. 1994. Proceedings--Workshop on Needle Exchange and Bleach Distribution Programs. Washington, DC: The National Academies Press. doi: 10.17226/4552.
×
Page 202
Suggested Citation:"EVALUATION METHODS." Institute of Medicine and National Research Council. 1994. Proceedings--Workshop on Needle Exchange and Bleach Distribution Programs. Washington, DC: The National Academies Press. doi: 10.17226/4552.
×
Page 203
Suggested Citation:"EVALUATION METHODS." Institute of Medicine and National Research Council. 1994. Proceedings--Workshop on Needle Exchange and Bleach Distribution Programs. Washington, DC: The National Academies Press. doi: 10.17226/4552.
×
Page 204
Suggested Citation:"EVALUATION METHODS." Institute of Medicine and National Research Council. 1994. Proceedings--Workshop on Needle Exchange and Bleach Distribution Programs. Washington, DC: The National Academies Press. doi: 10.17226/4552.
×
Page 205
Suggested Citation:"EVALUATION METHODS." Institute of Medicine and National Research Council. 1994. Proceedings--Workshop on Needle Exchange and Bleach Distribution Programs. Washington, DC: The National Academies Press. doi: 10.17226/4552.
×
Page 206
Suggested Citation:"EVALUATION METHODS." Institute of Medicine and National Research Council. 1994. Proceedings--Workshop on Needle Exchange and Bleach Distribution Programs. Washington, DC: The National Academies Press. doi: 10.17226/4552.
×
Page 207
Suggested Citation:"EVALUATION METHODS." Institute of Medicine and National Research Council. 1994. Proceedings--Workshop on Needle Exchange and Bleach Distribution Programs. Washington, DC: The National Academies Press. doi: 10.17226/4552.
×
Page 208
Suggested Citation:"EVALUATION METHODS." Institute of Medicine and National Research Council. 1994. Proceedings--Workshop on Needle Exchange and Bleach Distribution Programs. Washington, DC: The National Academies Press. doi: 10.17226/4552.
×
Page 209
Suggested Citation:"EVALUATION METHODS." Institute of Medicine and National Research Council. 1994. Proceedings--Workshop on Needle Exchange and Bleach Distribution Programs. Washington, DC: The National Academies Press. doi: 10.17226/4552.
×
Page 210
Suggested Citation:"EVALUATION METHODS." Institute of Medicine and National Research Council. 1994. Proceedings--Workshop on Needle Exchange and Bleach Distribution Programs. Washington, DC: The National Academies Press. doi: 10.17226/4552.
×
Page 211
Suggested Citation:"EVALUATION METHODS." Institute of Medicine and National Research Council. 1994. Proceedings--Workshop on Needle Exchange and Bleach Distribution Programs. Washington, DC: The National Academies Press. doi: 10.17226/4552.
×
Page 212
Suggested Citation:"EVALUATION METHODS." Institute of Medicine and National Research Council. 1994. Proceedings--Workshop on Needle Exchange and Bleach Distribution Programs. Washington, DC: The National Academies Press. doi: 10.17226/4552.
×
Page 213
Suggested Citation:"EVALUATION METHODS." Institute of Medicine and National Research Council. 1994. Proceedings--Workshop on Needle Exchange and Bleach Distribution Programs. Washington, DC: The National Academies Press. doi: 10.17226/4552.
×
Page 214
Suggested Citation:"EVALUATION METHODS." Institute of Medicine and National Research Council. 1994. Proceedings--Workshop on Needle Exchange and Bleach Distribution Programs. Washington, DC: The National Academies Press. doi: 10.17226/4552.
×
Page 215
Suggested Citation:"EVALUATION METHODS." Institute of Medicine and National Research Council. 1994. Proceedings--Workshop on Needle Exchange and Bleach Distribution Programs. Washington, DC: The National Academies Press. doi: 10.17226/4552.
×
Page 216
Suggested Citation:"EVALUATION METHODS." Institute of Medicine and National Research Council. 1994. Proceedings--Workshop on Needle Exchange and Bleach Distribution Programs. Washington, DC: The National Academies Press. doi: 10.17226/4552.
×
Page 217
Suggested Citation:"EVALUATION METHODS." Institute of Medicine and National Research Council. 1994. Proceedings--Workshop on Needle Exchange and Bleach Distribution Programs. Washington, DC: The National Academies Press. doi: 10.17226/4552.
×
Page 218
Suggested Citation:"EVALUATION METHODS." Institute of Medicine and National Research Council. 1994. Proceedings--Workshop on Needle Exchange and Bleach Distribution Programs. Washington, DC: The National Academies Press. doi: 10.17226/4552.
×
Page 219
Suggested Citation:"EVALUATION METHODS." Institute of Medicine and National Research Council. 1994. Proceedings--Workshop on Needle Exchange and Bleach Distribution Programs. Washington, DC: The National Academies Press. doi: 10.17226/4552.
×
Page 220
Suggested Citation:"EVALUATION METHODS." Institute of Medicine and National Research Council. 1994. Proceedings--Workshop on Needle Exchange and Bleach Distribution Programs. Washington, DC: The National Academies Press. doi: 10.17226/4552.
×
Page 221
Suggested Citation:"EVALUATION METHODS." Institute of Medicine and National Research Council. 1994. Proceedings--Workshop on Needle Exchange and Bleach Distribution Programs. Washington, DC: The National Academies Press. doi: 10.17226/4552.
×
Page 222
Suggested Citation:"EVALUATION METHODS." Institute of Medicine and National Research Council. 1994. Proceedings--Workshop on Needle Exchange and Bleach Distribution Programs. Washington, DC: The National Academies Press. doi: 10.17226/4552.
×
Page 223
Suggested Citation:"EVALUATION METHODS." Institute of Medicine and National Research Council. 1994. Proceedings--Workshop on Needle Exchange and Bleach Distribution Programs. Washington, DC: The National Academies Press. doi: 10.17226/4552.
×
Page 224
Suggested Citation:"EVALUATION METHODS." Institute of Medicine and National Research Council. 1994. Proceedings--Workshop on Needle Exchange and Bleach Distribution Programs. Washington, DC: The National Academies Press. doi: 10.17226/4552.
×
Page 225
Suggested Citation:"EVALUATION METHODS." Institute of Medicine and National Research Council. 1994. Proceedings--Workshop on Needle Exchange and Bleach Distribution Programs. Washington, DC: The National Academies Press. doi: 10.17226/4552.
×
Page 226
Suggested Citation:"EVALUATION METHODS." Institute of Medicine and National Research Council. 1994. Proceedings--Workshop on Needle Exchange and Bleach Distribution Programs. Washington, DC: The National Academies Press. doi: 10.17226/4552.
×
Page 227
Suggested Citation:"EVALUATION METHODS." Institute of Medicine and National Research Council. 1994. Proceedings--Workshop on Needle Exchange and Bleach Distribution Programs. Washington, DC: The National Academies Press. doi: 10.17226/4552.
×
Page 228
Suggested Citation:"EVALUATION METHODS." Institute of Medicine and National Research Council. 1994. Proceedings--Workshop on Needle Exchange and Bleach Distribution Programs. Washington, DC: The National Academies Press. doi: 10.17226/4552.
×
Page 229
Suggested Citation:"EVALUATION METHODS." Institute of Medicine and National Research Council. 1994. Proceedings--Workshop on Needle Exchange and Bleach Distribution Programs. Washington, DC: The National Academies Press. doi: 10.17226/4552.
×
Page 230
Suggested Citation:"EVALUATION METHODS." Institute of Medicine and National Research Council. 1994. Proceedings--Workshop on Needle Exchange and Bleach Distribution Programs. Washington, DC: The National Academies Press. doi: 10.17226/4552.
×
Page 231
Suggested Citation:"EVALUATION METHODS." Institute of Medicine and National Research Council. 1994. Proceedings--Workshop on Needle Exchange and Bleach Distribution Programs. Washington, DC: The National Academies Press. doi: 10.17226/4552.
×
Page 232
Suggested Citation:"EVALUATION METHODS." Institute of Medicine and National Research Council. 1994. Proceedings--Workshop on Needle Exchange and Bleach Distribution Programs. Washington, DC: The National Academies Press. doi: 10.17226/4552.
×
Page 233
Suggested Citation:"EVALUATION METHODS." Institute of Medicine and National Research Council. 1994. Proceedings--Workshop on Needle Exchange and Bleach Distribution Programs. Washington, DC: The National Academies Press. doi: 10.17226/4552.
×
Page 234
Suggested Citation:"EVALUATION METHODS." Institute of Medicine and National Research Council. 1994. Proceedings--Workshop on Needle Exchange and Bleach Distribution Programs. Washington, DC: The National Academies Press. doi: 10.17226/4552.
×
Page 235
Suggested Citation:"EVALUATION METHODS." Institute of Medicine and National Research Council. 1994. Proceedings--Workshop on Needle Exchange and Bleach Distribution Programs. Washington, DC: The National Academies Press. doi: 10.17226/4552.
×
Page 236
Suggested Citation:"EVALUATION METHODS." Institute of Medicine and National Research Council. 1994. Proceedings--Workshop on Needle Exchange and Bleach Distribution Programs. Washington, DC: The National Academies Press. doi: 10.17226/4552.
×
Page 237
Suggested Citation:"EVALUATION METHODS." Institute of Medicine and National Research Council. 1994. Proceedings--Workshop on Needle Exchange and Bleach Distribution Programs. Washington, DC: The National Academies Press. doi: 10.17226/4552.
×
Page 238
Suggested Citation:"EVALUATION METHODS." Institute of Medicine and National Research Council. 1994. Proceedings--Workshop on Needle Exchange and Bleach Distribution Programs. Washington, DC: The National Academies Press. doi: 10.17226/4552.
×
Page 239
Suggested Citation:"EVALUATION METHODS." Institute of Medicine and National Research Council. 1994. Proceedings--Workshop on Needle Exchange and Bleach Distribution Programs. Washington, DC: The National Academies Press. doi: 10.17226/4552.
×
Page 240
Suggested Citation:"EVALUATION METHODS." Institute of Medicine and National Research Council. 1994. Proceedings--Workshop on Needle Exchange and Bleach Distribution Programs. Washington, DC: The National Academies Press. doi: 10.17226/4552.
×
Page 241
Suggested Citation:"EVALUATION METHODS." Institute of Medicine and National Research Council. 1994. Proceedings--Workshop on Needle Exchange and Bleach Distribution Programs. Washington, DC: The National Academies Press. doi: 10.17226/4552.
×
Page 242
Suggested Citation:"EVALUATION METHODS." Institute of Medicine and National Research Council. 1994. Proceedings--Workshop on Needle Exchange and Bleach Distribution Programs. Washington, DC: The National Academies Press. doi: 10.17226/4552.
×
Page 243
Suggested Citation:"EVALUATION METHODS." Institute of Medicine and National Research Council. 1994. Proceedings--Workshop on Needle Exchange and Bleach Distribution Programs. Washington, DC: The National Academies Press. doi: 10.17226/4552.
×
Page 244
Suggested Citation:"EVALUATION METHODS." Institute of Medicine and National Research Council. 1994. Proceedings--Workshop on Needle Exchange and Bleach Distribution Programs. Washington, DC: The National Academies Press. doi: 10.17226/4552.
×
Page 245
Suggested Citation:"EVALUATION METHODS." Institute of Medicine and National Research Council. 1994. Proceedings--Workshop on Needle Exchange and Bleach Distribution Programs. Washington, DC: The National Academies Press. doi: 10.17226/4552.
×
Page 246
Suggested Citation:"EVALUATION METHODS." Institute of Medicine and National Research Council. 1994. Proceedings--Workshop on Needle Exchange and Bleach Distribution Programs. Washington, DC: The National Academies Press. doi: 10.17226/4552.
×
Page 247
Suggested Citation:"EVALUATION METHODS." Institute of Medicine and National Research Council. 1994. Proceedings--Workshop on Needle Exchange and Bleach Distribution Programs. Washington, DC: The National Academies Press. doi: 10.17226/4552.
×
Page 248
Suggested Citation:"EVALUATION METHODS." Institute of Medicine and National Research Council. 1994. Proceedings--Workshop on Needle Exchange and Bleach Distribution Programs. Washington, DC: The National Academies Press. doi: 10.17226/4552.
×
Page 249
Suggested Citation:"EVALUATION METHODS." Institute of Medicine and National Research Council. 1994. Proceedings--Workshop on Needle Exchange and Bleach Distribution Programs. Washington, DC: The National Academies Press. doi: 10.17226/4552.
×
Page 250
Suggested Citation:"EVALUATION METHODS." Institute of Medicine and National Research Council. 1994. Proceedings--Workshop on Needle Exchange and Bleach Distribution Programs. Washington, DC: The National Academies Press. doi: 10.17226/4552.
×
Page 251
Suggested Citation:"EVALUATION METHODS." Institute of Medicine and National Research Council. 1994. Proceedings--Workshop on Needle Exchange and Bleach Distribution Programs. Washington, DC: The National Academies Press. doi: 10.17226/4552.
×
Page 252
Next: BLEACH DISTRIBUTION PROGRAMS »
Proceedings--Workshop on Needle Exchange and Bleach Distribution Programs Get This Book
×
Buy Paperback | $55.00
MyNAP members save 10% online.
Login or Register to save!

This book reports on research on and experience with needle exchange and bleach distribution programs and their effects on rates of drug use, the behavior of injection drug users, and the spread of HIV and other infectious diseases among injection drug users. It discusses U.S. needle exchange data, international evaluations of needle exchange programs, legal issues and drug paraphernalia laws, evaluation methods, and bleach distribution programs.

  1. ×

    Welcome to OpenBook!

    You're looking at OpenBook, NAP.edu's online reading room since 1999. Based on feedback from you, our users, we've made some improvements that make it easier than ever to read thousands of publications on our website.

    Do you want to take a quick tour of the OpenBook's features?

    No Thanks Take a Tour »
  2. ×

    Show this book's table of contents, where you can jump to any chapter by name.

    « Back Next »
  3. ×

    ...or use these buttons to go back to the previous chapter or skip to the next one.

    « Back Next »
  4. ×

    Jump up to the previous page or down to the next one. Also, you can type in a page number and press Enter to go directly to that page in the book.

    « Back Next »
  5. ×

    Switch between the Original Pages, where you can read the report as it appeared in print, and Text Pages for the web version, where you can highlight and search the text.

    « Back Next »
  6. ×

    To search the entire text of this book, type in your search term here and press Enter.

    « Back Next »
  7. ×

    Share a link to this book page on your preferred social network or via email.

    « Back Next »
  8. ×

    View our suggested citation for this chapter.

    « Back Next »
  9. ×

    Ready to take your reading offline? Click here to buy this book in print or download it as a free PDF, if available.

    « Back Next »
Stay Connected!