Below are the first 10 and last 10 pages of uncorrected machine-read text (when available) of this chapter, followed by the top 30 algorithmically extracted key phrases from the chapter as a whole.
Intended to provide our own search engines and external engines with highly rich, chapter-representative searchable text on the opening pages of each chapter. Because it is UNCORRECTED material, please consider the following text as a useful but insufficient proxy for the authoritative book pages.
Do not use for reproduction, copying, pasting, or reading; exclusively for search engines.
OCR for page 129
Complementary and Alternative Medicine in the United States 5 State of Emerging Evidence on CAM For policy makers, practitioners, patients, and health care system managers to make informed decisions about the use of complementary and alternative (CAM) therapies, they must have both access to and a means of evaluating the results of research on the topic. This chapter discusses the evidence that forms the basis for such decision making and the methods of evaluation, as well as the available resources providing access to the results of existing research on CAM interventions. In CAM as well as in conventional medicine, randomized controlled trials (RCTs), when possible, are the preferable study design for assessing efficacy. RCTs use random allocation to create comparable groups, after which an intervention is introduced. This intervention consists of treatment for the test group and a placebo, no treatment, or another active treatment for the control group. Once the outcome is recorded, any observable differences between the treatment and control group should be attributable to the intervention because the groups were initially comparable before the intervention was introduced. A systematic review uses explicit, systematic methods to review existing research, particularly the effectiveness of health care interventions, as evaluated by RCTs. Some systematic reviews may include meta-analyses, which provide an overview of the results of similar studies by the use of statistical methods to evaluate the data from many studies. Systematic reviews are widely considered the best method for gathering and synthesizing evidence as well as for determining gaps that exist in current research. Although basic science research and evaluation of cost-effectiveness are also important aspects of research on CAM therapies and modalities, the focus of the
OCR for page 130
Complementary and Alternative Medicine in the United States following sections is the evaluation of the clinical efficacies of therapies by RCTs and systematic reviews. SOURCES OF INFORMATION ON HIGH-QUALITY EVIDENCE Two main sources of information about published RCTs and systematic reviews are The Cochrane Library and MEDLINE. Critical reviews of reviews and Agency for Healthcare Research and Quality (AHRQ) Evidence Reports summarize the information by using rigorous and objective methods. National Institutes of Health (NIH) Consensus Statements incorporate evidence from RCTs and systematic reviews together with the judgments of a panel of nonadvocate, nongovernmental experts, to reach a decision about the efficacy and safety of a particular treatment. MEDLINE MEDLINE, a product of the National Library of Medicine, is an extensive bibliographic database covering all areas of clinical medicine and biomedical research. The bibliographic citations and abstracts indexed in MEDLINE are from more than 4,600 biomedical journals published worldwide, and the database includes information on all randomized trials in MEDLINE-indexed journals, regardless of the methodological quality or clinical relevance. MEDLINE is accessible online and is free of charge to the public, and most of its 12 million citations are available in English, at least as abstracts. The database includes studies published since 1966, the year that MEDLINE began, and is updated on a regular basis (National Library of Medicine, 2002). In recent years, relevant indexing terms have been introduced on MEDLINE to facilitate queries on trials and systematic reviews related to CAM. MEDLINE introduced the publication type “randomized controlled trial” for specific RCTs in 1991 and the subject subset “systematic review” in 2001. The subject subset “CAM” was introduced in 2001 and includes all records identified through the execution of a complex, highly sensitive search strategy designed to identify all records in the MEDLINE database related to CAM. The introduction of these terms allows interested individuals to make simple queries of MEDLINE to estimate changes in the evidence base for CAM from the results of RCTs and systematic reviews over time. Figure 5-1 charts the tremendous growth in the number of RCTs over the past 20 years, and Figure 5-2 shows that the rate of increase of reviews and meta-analyses is even greater. These increases parallel general trends of growth in trials and meta-analyses over the past twenty years (Lee et al., 2001). Despite these developments, however, limitations of MEDLINE persist: not all studies in MEDLINE are indexed with the appropriate terms
OCR for page 131
Complementary and Alternative Medicine in the United States FIGURE 5-1 Number of CAM RCTs indexed on MEDLINE, 1982 to 2002. This search was performed on December 11, 2003, by using a search strategy with the following terms to obtain counts for each year: randomized controlled trial (publication type) AND year (publication date). FIGURE 5-2 Number of reviews and meta-analyses related to CAM indexed on MEDLINE, 1982 to 2002. This search was performed on December 11, 2003, by using a search strategy with the following two sets of terms to obtain counts for each year: systematic (subset type) AND year (publication date) and meta-analysis (publication type) AND year (publication date).
OCR for page 132
Complementary and Alternative Medicine in the United States (Dickersin et al., 1994), and many reports, especially in the field of CAM, are not included on MEDLINE (Egger et al., 2003). The Cochrane Library The Cochrane Library, unique both for its scope and for its methodological standards, is supported through the work of the Cochrane Collaboration (Dickersin and Manheimer, 1998), an international organization of more than 9,000 contributors (mostly volunteers) from more than 80 countries (Allen and Clarke, 2003). The Cochrane Complementary Medicine Field, based at the University of Maryland Center for Integrative Medicine, coordinates all of the CAM-related activities of the Cochrane Collaboration, including the development of a database with information on more than 7,000 controlled trials of CAM therapies and modalities, and facilitates the preparation of CAM reviews and the promotion of these reviews, especially among the members of the CAM community. Members of the Cochrane Collaboration prepare up-to-date, reliable summaries or systematic reviews of every kind of health care therapy. Cochrane reviews, which are intended to answer questions about health care and to guide providers in practical decision making about treatment, are published quarterly in The Cochrane Library. Although reviews of journal articles are current only as of their date of publication, the electronic format of The Cochrane Library allows the reviews to be updated easily and periodically to account for new evidence. In addition, Cochrane reviews have shown greater methodological rigor than systematic reviews and meta-analyses published in paper-based journals (Jadad et al., 1998, 2000). The use of rigorous methods is also ensured by the requirement that the protocols used for a review be prepared before a review is conducted and by an extensive system of peer review. Because the information in The Cochrane Library is prescreened to a certain extent and includes only studies evaluating health care therapies, it is of generally higher quality and greater relevance to patient care than the information available on MEDLINE. The Cochrane Database of Systematic Reviews and the Cochrane Central Register of Controlled Trials comprise the major databases of the Cochrane Collaboration. If no review is available on the Cochrane Database of Systematic Reviews, one can check The Cochrane Library’s Database of Abstracts of Reviews of Effectiveness (a database of reviews collected by the Cochrane Collaboration but not prepared according to the strict standards of Cochrane reviews) or the applicable trials registered with the Cochrane Central Register of Controlled Trials. The Cochrane database of CAM-related clinical trials is an immensely valuable resource because it covers many of the controlled trials that would
OCR for page 133
Complementary and Alternative Medicine in the United States not be included on MEDLINE. Vickers (1998) analyzed the Cochrane register for studies related to CAM and found that 19 percent of the citations were not indexed on MEDLINE. The trials included in the register were derived from 965 different journals; 84 percent of the trials were published in conventional medical journals. The numbers of trials per therapy varied a great deal: although Vickers found 554 trials of acupuncture and 804 of herbal medicine, he retrieved only 47 trials of aromatherapy. The number of trials per condition also varied, with 501 trials of cardiovascular disease, 386 trials of musculoskeletal disorders, and 293 trials of surgery-related symptoms, but only 11 trials of fatigue disorders. The objective of the register is to include all large multicenter trials, such as those recently published showing that St. John’s wort and echinacea are ineffective for the treatment of major depression and the common cold, respectively (Hypericum Depression Trial Study Group, 2002; Taylor et al., 2003). Also included are smaller RCTs, such as pilot studies of acupuncture conducted in China. The ultimate aim of developing the Cochrane CAM register is to provide a comprehensive source of trials of CAM therapies and modalities, thus reducing the need for systematic reviewers and others to search multiple sources. At present The Cochrane Library contains 145 CAM-related systematic and an additional 340 non-Cochrane CAM-related systematic reviews (see Table 5-1 for a sampling of therapies covered by Cochrane and non-Cochrane reviews). These reviews cover many areas of CAM, with particular strength in the fields of acupuncture and herbal medicine, reflecting not only the large number of trials in these fields, but also the great interest of clinicians, policy makers, and consumers in these areas. There are some disparities between evidence from Cochrane reviews and evidence from clinical practice. For example, although relaxation techniques (e.g., meditation) are the most commonly used CAM therapy among the U.S. general population (Eisenberg et al., 1998) and the fourth most commonly used therapy in hospital-based CAM or wellness centers (Health Forum, 2003), few Cochrane reviews have evaluated such therapies. On the other hand, although herbal therapy and treatment with other dietary supplements are not widely offered in U.S. hospitals, they are the most reviewed and are among the therapies that are the most commonly used by the U.S. public (Eisenberg et al., 1998). The international structure of the Cochrane Collaboration plays a critical role in the identification of CAM trials and the preparation of reviews of CAM treatments and modalities because the therapies considered CAM in the United States are often the traditional medicines used by the populations of other countries. Through the work of the 14 Cochrane Centers worldwide, journals published around the world are hand searched to identify RCTs on conventional medicine therapies and CAM therapies and
OCR for page 134
Complementary and Alternative Medicine in the United States TABLE 5-1 Number of Cochrane and Non-Cochrane Reviews, by Therapy, March 2004 Therapy Cochrane Reviews Non-Cochrane Reviews Acupuncture 10 69 Alexander technique 1 0 Art therapy 1 1 Biofeedback 2 26 Chiropractic 2 33 Dietary supplements (nonherbal) 71 46 Electromagnetic therapy 3 11 Herbal therapy 23 79 Homeopathy 4 34 Laser therapy 4 4 Massage therapy 4 18 Prayer 1 2 Transcutaneous electrical nerve stimulation 7 11 Therapeutic touch 1 3 Yoga 2 4 Other 27 91 NOTE: Some reviews cover multiple therapies and are therefore counted multiple times. The total number of Cochrane and Non-Cochrane reviews represented in this table are 156 and 340, respectively. modalities that may be relevant and eligible for a systematic review. The Chinese Cochrane Centre, for example, has identified an estimated 10,000 trials of traditional Chinese medicine through their hand searches (Tang and Wong, 1998); moreover, dozens of reviews of traditional Chinese medicine are under way. Cochrane Review Evidence for CAM1 All Cochrane reviews, be they of CAM or conventional medicine therapies, apply the same standards, that is, therapies within both categories are 1 The committee did not include information about the general direction of effect for the AHRQ reports because the individual reports covered too wide a range of conditions (e.g., S-Adenosyl-L-Methionine (SAMe) for Depression, Osteoarthritis, and Liver Disease) and therapies (e.g., Mind-Body Interventions for Gastrointestinal Conditions). Cochrane reviews, in contrast, typically evaluate a specific therapy for a specific condition. Concise summaries of the findings of each of the AHRQ Technology Reports are presented on the AHRQ website or http://www.ahrq.gov/clinic/epcindex.htm#complementary.
OCR for page 135
Complementary and Alternative Medicine in the United States evaluated according to the strength of evidence from RCTs. To evaluate the evidence for CAM from Cochrane reviews, all reviews of CAM-related therapies were selected from The Cochrane Library and assigned categories, as described below. As a means of applying an objective, reproducible, and operational eligibility criterion, the committee considered Cochrane reviews to be related to CAM only if the therapies reviewed were listed as therapies in the National Center for Complementary and Alternative Medicine (NCCAM)-National Library of Medicine CAM on PubMed, a database of abstracts and articles on CAM-related therapies. The database can be accessed by use of a multipage search strategy designed to identify all studies listed on PubMed that should be indexed in the PubMed CAM subset. The results from all eligible Cochrane reviews of CAM therapies were assigned to one of the following six categories by two trained methodologists: positive effect, possibly positive effect, two active treatments are equal, insufficient or inconclusive evidence of an effect, no effect, or harmful effect. When the two raters differed on their classification of the treatment described in a review, a third rater trained in RCT and systematic review methodologies assigned the final classification. This rating system was used in a previous study to assess the evidence base for conventional medicine according to the information found in Cochrane reviews (Ezzo et al., 2001). The agreement of the classification assignment between the initial two raters was 83 percent. For the 17 percent of reviews for which the initial raters assigned different classification codes, the third rater agreed with one of the initial two raters’ codes in all cases. The largest number of treatments described in the reviews were classified as insufficient evidence of an effect (n = 82; 56.6 percent), followed by positive effect (n = 36; 24.8 percent) and possibly positive effect (n = 18; 12.4 percent). Only one review described a treatment that was classified as harmful (Caraballoso et al., 2003) (see Table 5-2). The reviews describing treatments classified as having positive effects are listed in Table 5-3. Although this exercise suggests that there is strong evidence for the effectiveness of some CAM therapies, much more research is required, as demonstrated by the large proportion of reviews of treatments classified as insufficient evidence of an effect. The fact that only one of the treatments in the Cochrane reviews fell into the harmful effect category suggests that clinical trials of CAM therapies have posed little risk to the participants. Some interesting findings emerge when the results of the evaluation of Cochrane reviews of CAM therapies are compared with the results of the earlier study (Ezzo et al., 2001) evaluating Cochrane reviews of conventional therapies: insufficient evidence of an effect was determined for a larger proportion of CAM therapies (56.6 percent for CAM versus 21.3
OCR for page 136
Complementary and Alternative Medicine in the United States TABLE 5-2 Conclusion Categories, Definitions, and Proportions Classified by Readers for Cochrane CAM Reviews Conclusion Category Definition Readers’ Consensual Rating for Included Reviews (n = 145) (%) Positive effect Treatment is more beneficial/effective than control for the primary outcome. 36 (24.8) Possibly positive effect Treatment may have a positive effect, but a major unresolved methodological issue, such as all studies being very low quality, or findings based on only one study, precludes making a definitive statement. 18 (12.4) Two active treatments are equal Two biologically active treatments, such as two antibiotics, are equally as effective for the condition being treated. This category to be used only when comparing two active treatments, not placebo or no treatment. 1 (.6) Insufficient/ inconclusive evidence There is not sufficient evidence to determine effectiveness. 82 (56.6) No effect There is sufficient evidence, and there is no effect. 7 (4.8) Harmful effect Treatment does more harm than good. 1 (.6) percent for conventional medicine), CAM therapies were less likely to be classified as harmful (8.1 percent for conventional medicine versus 0.69 percent for CAM) or as having no effect (20.0 percent for conventional medicine versus 4.8 percent for CAM), and classification of the therapies as having positive or a possibly positive effect was approximately equal for CAM and conventional medicine therapies (41.3 percent for conventional medicine versus 38.4 percent for CAM). In making comparisons between the two studies, however, it is important to keep in mind that the studies were conducted at different times and thus included different sets of Cochrane reviews. The study of Ezzo et al. (2001) included only those reviews published in The Cochrane Library at the time of its first issue in 1998, whereas the analysis of CAM described above included reviews published in the 2004 issue of the database, which is now much more comprehensive and better developed.
OCR for page 137
Complementary and Alternative Medicine in the United States TABLE 5-3 Cochrane CAM Reviewsa with Positive Effects Study Therapy Indicationb Limitation/ Commentc Atallah AN et al., 2004 Calcium Preventing hypertensive disorders and related problems in pregnancy 1, 4 Beckles WN et al., 2004 Omega-3 fatty acids (from fish oil) Cystic fibrosis 1, 2 Brosseau L et al., 2004 Transcutaneous electrical nerve stimulation (TENS) Rheumatoid arthritis in the hand 1 D’Souza RM and D’Souza R, 2004 Vitamin A Measles 4 Darlow B and Austin N, 2004 Selenium Short-term morbidity in preterm neonates 4 Darlow BA and Graham PJ, 2004 Vitamin A Preventing morbidity and mortality in very low-birthweight infants 1, 4 Douglas RM et al., 2004 Vitamin C Preventing and treating the common cold 1 (does not prevent colds; only reduces duration of symptoms) Evans JR, 2004 Antioxidant vitamin and mineral supplements Age-related macular degeneration 4, 5 Farmer A et al., 2004 Fish oil Type 2 diabetes mellitus 3, 5 Furlan AD et al., 2004 Massage Low back pain 2, 6 Herxheimer A and Petrie KJ, 2004 Melatonin Preventing and treating jet lag 5
OCR for page 138
Complementary and Alternative Medicine in the United States Study Therapy Indicationb Limitation/ Commentc Homik J et al., 2004 Calcium and vitamin D Corticosteroid-induced osteoporosis — Howlett A and Ohlsson A, 2004 Inositol Respiratory distress syndrome in preterm infants — Hulme J et al., 2004 Electromagnetic fields Osteoarthritis — Jepson RG et al., 2004 Cranberries Preventing urinary tract infections 1 Linde K and Mulrow CD, 2004 St. John’s wort Depression 2 Little CV and Parsons T, 2004 Herbal therapy Osteoarthritis —(only one herb found effective) Lumley J et al., 2004 Periconceptual supplementation with folate and/or multivitamins Preventing neural tube defects 4 Mahomed K, 2004 Folate supplementation in pregnancy Biochemical parameters and pregnancy outcome 3, 4 Mahomed K, 2004 Iron and folate supplementation in pregnancy Biochemical parameters and pregnancy outcome 3, 4 Mahomed K, 2004 Iron supplementation in pregnancy Biochemical parameters and pregnancy outcome 3, 4 Melchart D et al., 2004 Acupuncture Headache — Melchart D et al., 2004 Echinacea Preventing and treating the common cold 1 Ortiz Z et al., 2004 Folic acid and folinic acid Reducing side effects in patients receiving methotrexate 1
OCR for page 139
Complementary and Alternative Medicine in the United States Study Therapy Indicationb Limitation/ Commentc Osiri M et al., 2004 TENS Knee osteoarthritis 1 Pittler MH and Ernst E, 2004 Horse chestnut Chronic venous insufficiency — Pittler MH and Ernst E, 2004 Kava Anxiety 5 Proctor ML et al., 2004 TENS and acupuncture Primary dysmenorrhoea 1 Shea B et al., 2004 Calcium Bone loss — Taylor MJ et al., 2004 Folate Depression 4 Towheed TE et al., 2004 Glucosamine Osteoarthritis 1, 2 Wilson ML and Murphy PA, 2004 Herbal and dietary therapies Primary and secondary dysmenorrhoea — Wilt T et al., 2004 African prune Benign prostatic hyperplasia 1, 2 Wilt T et al., 2004 Beta-sitosterols Benign prostatic hyperplasia 2 Wilt T et al., 2004 Saw palmetto Benign prostatic hyperplasia 2 Wilt T et al., 2004 Cernilton Benign prostatic hyperplasia 1, 2, 6 aOur application of the list of terms used in the CAM on PubMed search strategy in defining our eligibility criteria for CAM-related resulted in the inclusion in our sample of some Cochrane reviews that many would not consider to be CAM-related. It is notoriously difficult to make the determination of whether or not a therapy should be considered CAM-related, and this decision must often be made in the context of the therapy’s application (e.g., for nutrients, whether it is used as a supplement or to address a deficiency); the setting in which the therapy is used (e.g., hospital, self-care); the current state of the evidence for the therapy (e.g., a therapy such as folic acid for neural tube defects has strong supportive evidence which has resulted in its integrated into the health care system); and the point of historical time at which the evaluation of the therapy as CAM or not CAM is made. Our CAM eligibility criteria, while initially deemed the most objective and reproducible approach for selecting CAM reviews, does not take into account the indication for which the therapy is used and has resulted in our including multiple reviews that are not CAM (e.g., vitamin A for measles) and excluding other reviews that are CAM (e.g., speleotherapy for the treatment of asthma), but the therapy reviewed is not a term in the CAM on PubMed search strategy. Because indexing reports always involves some degree of subjectivity (especially when indexing on a difficult-to-pin-down term such as CAM) and because the CAM on PubMed search strategy still requires improvements in the recall and precision of its terms, a second review, by an authority in the field, of reviews that were possibly inappropriately included or excluded based on the strict application of the CAM on PubMed search strategy terms, may be necessary.
OCR for page 157
Complementary and Alternative Medicine in the United States fosters rapid research of high quality. The third type of network is referred to as “whole systems,” in which the leadership is multidisciplinary. Network participants form coalitions including both research experts and novices. According to Thomas et al. (2001), the whole systems approach is good for producing cultural change because different enthusiasts in different parts of the health care system become involved. Griffiths and colleagues (2000) describe different network organizational styles in the following manner: … some have a hierarchical organisation with a strong centre, often at a university, leading satellite units or network members; others are less hierarchical with coordination and cooperation between satellite units and members as well as with the centre. Despite differences in design and organization, Nutting et al. (1999) describe four central characteristics of all PBRNs: PBRNs capture health care events that reflect the selection and observer bias that characterize primary care in community-based patient populations. PBRNs provide access to the practice experience and care provided by full-time primary care clinicians. PBRNs focus their activities on practice-relevant research questions, apply appropriate multimethod research designs, and generally avoid the tendency to permit research methods to define the question. Networks strive for the systematic involvement of network clinicians in defining the research questions, participating in the study design, and interpreting study results. PBRNs conduct studies that use both qualitative and quantitative methods ranging from observational studies to RCTs. For example, using observational data from The Direct Observation of Primary Care Study, Stange et al. (1998) found “that family physicians target preventive services toward patients most in need of them and use illness visits as opportunities for prevention”; a randomized control trial by Fleming et al. “provided the first direct evidence that brief physician intervention was effective in reducing alcohol use and utilization of health care services” (Nutting et al., 1999). Some networks focus on providing epidemiological data; others are concerned with the process of care (Griffiths et al., 2000). A PBRN can provide information on the content and the practice patterns offered in various types of clinical settings, offer flexibility in collecting and analyzing data from a variety of perspectives (e.g., the practitioner and the patient), provide the opportunity to ask and answer practice-relevant questions, and study CAM treatments in the manner in which they
OCR for page 158
Complementary and Alternative Medicine in the United States are practiced. Once a PBRN is in place, the additional costs of mounting specific studies can be fairly low, and the level of preparedness engendered by its structure can allow the timely generation of research findings (Zarin et al., 1997). An additional benefit of PBRNs, especially in the area of CAM, which has historically lacked a research infrastructure, is their potential to provide places of learning, provide training in research, and, through direct involvement of practitioners with science, promote a climate of inquiry that both questions and increases the evidence underlying a particular practice. Furthermore, according to Genel and Dobs (2003), PBRNs can facilitate the translation of research findings into practice. They assert that PBRNs address two of the greatest difficulties in translating findings into practice: the lack of communication between academic and practicing physicians and the failure to address practitioner needs in research. Genel and Dobs assert that practitioners must be trained in clinical research and that, as their familiarity with that research grows, they will be enthusiastic about the research effort and will be more likely to implement the research results in their own practices. CAM Research Centers The fourth major component of the research model proposed here is a CAM Research Center. Currently, NCCAM funds specialized research centers, each of which focuses on research in one particular area (see Appendix F for a list of these centers). For example, the center at the Oregon Health Sciences University focuses on CAM research in neurological disorders; the center at Columbia University investigates CAM use in aging; the center at the University of California at Los Angeles fosters research evaluating safety and efficacy of botanical dietary supplements; and the center at the University of Illinois at Chicago is studying botanical dietary supplements for women’s health. The CAM research centers envisioned in the committee’s research framework differ from these specialized research centers in important ways. First, they are not restricted to one central focus. Rather they would facilitate the activities of the research networks across many topic areas. The centers in Figure 5-3 would propose (or seek input from the network for) specific research questions and protocols. They would help coordinate the design and implementation of these investigations; coordinate refinement of protocols and priorities over time; and supervise the analyses of data generated from these studies. Unlike NCCAM centers as currently structured and funded, these new centers require a much broader spectrum of expertise from both the conventional and CAM research and clinical communities because they would not be focused on one modality or one clinical
OCR for page 159
Complementary and Alternative Medicine in the United States problem. These centers must have the capacity to work with the networks to identify important questions and to design studies that are hypothesis generating and hypothesis testing. The proposed centers coordinate data collection and analysis—bringing in information gleaned through longitudinal and periodic studies as well as data collected by the surveillance sites—and provide research and other training to network and surveillance participants as needed. In some instances, it is likely that they will serve in the more traditional role of a coordinating site for multisite observational or controlled trials, but this is not the main or sole purpose of their creation. The committee believes strongly that the center team should be transdisciplinary, at a minimum composed of methodologists, social and behavioral scientists, and experienced integrative medicine practitioners. As the network evolves it is anticipated that individuals from other disciplines would join the coordinating team. This team should be committed to a participatory research process and the cannons of conducting good science. These four major components, national surveys, a sentinel surveillance system, practice-based research networks, and CAM research centers, form the core of the research, reporting, and translation model developed by the committee and illustrated in Figure 5-3. A Model for CAM Research and Surveillance In the model for CAM research and surveillance shown in Figure 5-3 the left-hand hub represents the sentinel surveillance function. Sentinel surveillance sites would be responsible for the systematic collection and reporting of data on common and emerging patterns of CAM use as well as the use of both CAM and conventional medicine. Such information could be used to identify treatment trends that demand research. A sentinel surveillance approach to the collection of data on the use of CAM would complement the periodic population-based survey approach because data would be collected in an ongoing fashion in contexts subject to real-world contingencies and for a variety of different populations. The sentinel surveillance systems formed in different parts of the country would also provide data that allow the monitoring and analysis of regional and national trends in CAM use. Another advantage of sentinel surveillance systems relates to the translation of research results into practice. That is, such systems allow the real-time monitoring of the impact of the information about various CAM treatments disseminated to the public and practitioners by asking such questions as How does information influence practice? and How does treatment affect behaviorial change? The information collected by the sentinel sites could then be reported to the CAM research center (center of
OCR for page 160
Complementary and Alternative Medicine in the United States Figure 5-3), where it would be reviewed to determine whether emerging patterns that would be useful to study may exist. The right-hand hub of Figure 5-3 is the PBRN. Network participants would be recruited from many disciplines and would include both CAM and conventional medicine practitioners. It is anticipated that this network will conduct practice-based participatory research. Practitioners involved in the PBRN will learn research study designs through their preparation for and participation in network search activities. Additionally, these practitioners will be in a position to guide the development of new instruments and outcome measures of relevance to the community. In the model illustrated in Figure 5-3, the CAM research center would work with the PBRN to develop a consensus on how best to test the effectiveness and safety of treatments and bundles of behavior associated with treatment approaches. Together, the center and the network would Select target conditions to be evaluated Develop a protocol designed to capture Health care practices engaged in by patients as self-care Health care administered or prescribed by practitioners Interactions between self-care and practitioner care and between conventional medical care and CAM care Notable lifestyle changes that may effect health status, e.g., interactions between medication use and special diets the population has adopted Exposure of the population to marketing and public health messages Identify sites attending to patients (by both CAM and conventional medicine) with the targeted conditions Organize the data collection on the basis of an agreed-upon sampling procedure Develop an effectiveness study or intervention activity linked to the surveillance data It is the committee’s belief that the model for research and translation illustrated and described above would provide a coordinated mechanism directed at answering the myriad questions about CAM use, such as Who is using CAM and why are they doing so? and Are CAM therapies safe, effective, and cost-effective? The committee strongly urges NIH and other public agencies to provide the support necessary to develop and implement such a model.
OCR for page 161
Complementary and Alternative Medicine in the United States CONCLUSIONS AND RECOMMENDATIONS Both Chapter 2 and this chapter have presented information on what is known about CAM and where the gaps in knowledge exist. Chapter 2 presented the committee’s recommendations on ways to address the gaps in knowledge discussed in that chapter. These gaps in knowledge are reviewed below: The information available about the motivations for CAM use indicates that pursuit of wellness is a major impetus; however, the extent to which CAM use is a trigger for positive behavioral change is unknown and constitutes an important research issue. Existing surveys provide little information about how the use of CAM therapies is initiated; that is, are they self-initiated, provider initiated, provider administered, or self-administered? There are virtually no data on adherence to CAM treatment or self-treatment with CAM. This information is crucial to assessments of the real-world effectiveness and safety of CAM therapies and their use. Longitudinal studies are needed to clarify people’s trajectories of CAM use and those factors that influence upward and downward slopes in use. There is little research on how the public obtains information about CAM therapies: what types of information are deemed credible, marginal, and spurious; how does the public understand the information in terms of risks and benefits; how do such perceptions inform decision making; and what do members of the public expect their providers to tell them? This chapter has discussed the emerging evidence about CAM therapies, including sources of information (MEDLINE, The Cochrane Library); summarized the systematic reviews that have been conducted; examined the need for high-quality studies; and explored the gaps in evidence. The gaps discussed include the paucity of clinical research in which CAM interventions are compared with each other and with conventional medicine therapies, the need for expanded basic research to include areas other than botanicals, the lack of cost-effectiveness research, the need for cross-disciplinary and transdisciplinary research, and the importance of drawing established scientists to the field of CAM research. The committee has proposed a research framework to address these gaps as well as conduct the kinds of research recommended in Chapter 2. This framework includes a sentinel surveillance system, PBRNs, and CAM research centers that can incorporate information from national surveys (both periodic and longitudinal) and facilitate the work of the PBRNs.
OCR for page 162
Complementary and Alternative Medicine in the United States The committee believes that a research model such as the one described in this chapter, if it is adequately funded and implemented, will help provide additional understanding of the vast and varied field of CAM. As part of the following recommendation, the committee specifies the need for oversampling of racial and ethnic minorities in surveys. Many racial and ethnic minorities constitute a comparatively small proportion of the total U.S. population or are concentrated in a small number of geographic areas. In both situations, the number of individuals in a specific racial or ethnic group who will be selected in a nationwide random sample is typically too small to provide the basis for statistically reliable estimates for that population. Therefore, the only way to obtain meaningful results for such minority groups and to allow comparisons with the majority non-Hispanic white population is to oversample these groups, that is, to select the sample in a fashion that ensures that the proportion of individuals from these minority groups in the sample is larger than their proportion in the overall U.S. population. The committee recommends that the National Institutes of Health and other public agencies provide the support necessary to develop and implement a sentinel surveillance system, practice-based research networks, and CAM research centers to facilitate the work of the networks; include CAM-relevant questions in federally funded health care surveys (e.g., the National Health Interview Survey) and in ongoing longitudinal cohort studies (e.g., the Nurses’ Health Study and the Framingham Heart Study); and implement periodic comprehensive, representative, national surveys to assess the changes in the prevalence, patterns, perceptions, and costs of therapy use (both CAM and conventional medicine), with oversampling of ethnic minorities. REFERENCES AHRQ (Agency for Healthcare Research and Quality). 2001. Primary Care Practice-Based Research Networks. [Online]. Available: http://www.ahrq.gov/research/pbrnfact.htm [accessed June 16, 2004]. Allen C, Clark M. 2003. International Activity Within Collaborative Review Groups. In: 11th Cochrane Colloquium: Evidence, Healthcare and Culture. Oxford, UK: U.K. Cochrane Center. P. 44. Atallah AN, Hofmeyr GJ, Duley L. Calcium supplementation during pregnancy for preventing hypertensive disorders and related problems. Cochrane Database Syst Rev 2004; (1). Barnes PM, Powell-Griner E, McFann K, Nahin RL. 2004. Complementary and alternative medicine use among adults: United States, 2002. Vital Health Stat 343:1–19 (advance data).
OCR for page 163
Complementary and Alternative Medicine in the United States Bausell RB, Lee WL, Li YF, Soeken KS, Berman BM. 2004. Larger effect sizes were associated with higher quality ratings in complementary and alternative medicine randomized controlled trials. J Clin Epidemiol 57(5):438–446. Beckles Willson N, Elliott TM, Everard ML. Omega-3 fatty acids (from fish oils) for cystic fibrosis. Cochrane Database Syst Rev 2004; (1). Birkhead GS, Maylahn CM. 2000. State and Local Public Health Surveillance in Principles and Practice of Public Health Surveillance. Teutsch SM, Churchill RE, eds. New York: Oxford University Press. Pp. 253–287. Brosseau L, Yonge KA, Robinson V, Marchand S, Judd M, Wells G, Tugwell P. Transcutaneous electrical nerve stimulation (TENS) for the treatment of rheumatoid arthritis in the hand. Cochrane Database Syst Rev 2004; (1). Busby MG, Jeffcoat AR, Bloedon LT, Koch MA, Black T, Dix KJ, Heizer WD, Thomas BF, Hill JM, Crowell JA, Zeisel SH. 2002. Clinical characteristics and pharmacokinetics of purified soy isoflavones: Single-dose administration to healthy men. Am J Clin Nutr 75(1):126–136. Caraballoso M, Sacristan M, Serra C, Bonfill X. 2003. Drugs for preventing lung cancer in healthy people. Cochrane Database Syst Rev 2:CD002141. Caraballoso M, Sacristan M, Serra C, Bonfill X. Drugs for preventing lung cancer in healthy people. Cochrane Database Syst Rev 2004; (1). D’Souza RM, D’Souza R. Vitamin A for treating measles in children. Cochrane Database Syst Rev 2004; (1). Darlow B, Austin N. Selenium supplementation to prevent short-term morbidity in preterm neonates. Cochrane Database Syst Rev 2004; (4). Darlow BA, Graham PJ. Vitamin A supplementation for preventing morbidity and mortality in very low birthweight infants. Cochrane Database Syst Rev 2004; (4). Davidson RJ, Kabat-Zinn J, Schumacher J, Rosenkranz M, Muller D, Santorelli SF, Urbanowski F, Harrington A, Bonus K, Sheridan JF. 2003. Alterations in brain and immune function produced by mindfulness meditation . Psychosom Med 65(4):564–570. Dickersin K, Manheimer E. 1998. The Cochrane Collaboration: Evaluation of health care and services using systematic reviews of the results of randomized controlled trials. Clin Obstet Gynecol 41(2):315–331. Dickersin K, Scherer R, Lefebvre C. 1994. Identifying relevant studies for systematic reviews. BMJ 309(6964):1286–1291. Douglas RM, Chalker EB, Treacy B. Vitamin C for preventing and treating the common cold. Cochrane Database Syst Rev 2004; (1). Egger M, Juni P, Bartlett C, Holenstein F, Sterne J. 2003. How important are comprehensive literature searches and the assessment of trial quality in systematic reviews? Empirical study. Health Technol Assess 7(1):1–76. Eisenberg DM, Davis RB, Ettner SL, Appel S, Wilkey S, Van Rompay M, Kessler RC. 1998. Trends in alternative medicine use in the United States, 1990-1997: Results of a follow-up national survey. JAMA 280(18):1569–1575. Evans JR. Antioxidant vitamin and mineral supplements for age-related macular degeneration. Cochrane Database Syst Rev 2004; (1). Ezzo J, Bausell B, Moerman DE, Berman B, Hadhazy V. 2001. Reviewing the reviews. How strong is the evidence? How clear are the conclusions? Int J Technol Assess Health Care 17(4):457–466. Farmer A, Montori V, Dinneen S, Clar C. Fish oil in people with type 2 diabetes mellitus. Cochrane Database Syst Rev 2004; (1). Fenton E, Harvey J, Griffiths F, Wild A, Sturt J. 2001. Reflections from organization science on the development of primary health care research networks. Fam Pract 18(5):540–544.
OCR for page 164
Complementary and Alternative Medicine in the United States FPBRN (Federation of Practice Based Research Networks). 2004. Federation of Practice Based Research Networks Homepage. [Online]. Available: http://www.aafp.org/x19544.xml [accessed April 14, 2004]. Furlan AD, Brosseau L, Imamura M, Irvin E. Massage for low back pain. Cochrane Database Syst Rev 2004; (1). Friedman R, Sobel D, Myers P, Caudill M, Benson H. 1995. Behavioral medicine, clinical health psychology, and cost offset. Health Psychol 14(6):509–518. Genel M, Dobs A. 2003. Translating clinical research into practice: Practice-based research networks—a promising solution. J Investig Med 51(2):64–71. Green LA, Dovey SM. 2001. Practice based primary care research networks. They work and are ready for full development and support. BMJ 322(7286):567–568. Greenland P, Knoll MD, Stamler J, Neaton JD, Dyer AR, Garside EB, Wilson PW. 2003. Major risk factors as antecedents of fatal and nonfatal coronary heart disease events. JAMA 290:891–897. Griffiths F, Wild A, Harvey J, Fenton E. 2000. The productivity of primary care research networks. Br J Gen Pract 50(460):913–915. Grol R, Grimshaw J. 2003. From best evidence to best practice: Effective implementation of change in patients’ care. Lancet 362(9391):1225–1230. Groseclose SL, Sullivan KM, Gibbs NP, Knowles CM. 2000. Management of the Surveillance Information System and Quality Control of Data. In: Teutsch SM, Churchill RE, eds. Principles and Practice of Public Health Surveillance. New York: Oxford University Press. Pp. 95–112. Health Forum/American Hospital Association. 2003. 2003 Complementary and Alternative Medicine Survey. [Online]. Available: http://www.hospitalconnect.com/aha/resource_center/statistics/Complementary%20and%20Alternative%20Medicine%20Survey.html [accessed June 16, 2004]. Herxheimer A, Petrie KJ. Melatonin for preventing and treating jet lag. Cochrane Database Syst Rev 2004; (1). Homik J, Suarez-Almazor ME, Shea B, Cranney A, Wells G, Tugwell P. Calcium and vitamin D for corticosteroid-induced osteoporosis. Cochrane Database Syst Rev 2004; (1). Howlett A, Ohlsson A. Inositol for respiratory distress syndrome in preterm infants. Cochrane Database Syst Rev 2004; (1). Hulme J, Robinson V, DeBie R, Wells G, Judd M, Tugwell P. Electromagnetic fields for the treatment of osteoarthritis. Cochrane Database Syst Rev 2004; (1). Hypericum Depression Trial Study Group. 2002. Effect of Hypericum perforatum (St John’s wort) in major depressive disorder: A randomized controlled trial. JAMA 287(14):1807–1814. IOM (Institute of Medicine). 1996. Primary Care: America’s Health in a New Era. Washington, DC: National Academy Press. IOM. 2003a. The Future of the Public’s Health. Washington, DC: The National Academies Press. IOM. 2003b. Who Will Keep the Public Healthy? Washington, DC: The National Academies Press. Jadad AR, Moore RA, Carroll D, Jenkinson C, Reynolds DJ, Gavaghan DJ, McQuay HJ. 1996. Assessing the quality of reports of randomized clinical trials: Is blinding necessary? Control Clin Trials 17(1):1–12. Jadad AR, Cook DJ, Jones A, Klassen TP, Tugwell P, Moher M, Moher D. 1998. Methodology and reports of systematic reviews and meta-analyses: A comparison of Cochrane reviews with articles published in paper-based journals. JAMA 280(3):278–280.
OCR for page 165
Complementary and Alternative Medicine in the United States Jadad AR, Moher M, Browman GP, Booker L, Sigouin C, Fuentes M, Stevens R. 2000. Systematic reviews and meta-analyses on treatment of asthma: Critical evaluation. BMJ 320(7234):537–540. Jepson RG, Mihaljevic L, Craig J. Cranberries for preventing urinary tract infections. Cochrane Database Syst Rev 2004; (1). Juni P, Altman DG, Egger M. 2001. Systematic reviews in health care: Assessing the quality of controlled clinical trials. BMJ 323(7303):42–46. Kennelly EJ, Baggett S, Nuntanakorn P, Ososki AL, Mori SA, Duke J, Coleton M, Kronenberg F. 2002. Analysis of thirteen populations of black cohosh for formononetin. Phytomedicine 9(5):461–467. Lee WL, Bausell RB, Berman BM. 2001. The growth of health-related meta-analyses published from 1980 to 2000. Evaluation and the Health Professions 24(3):327–335. Lindbloom EJ, Ewigman BG, Hickner JM. 2004. Practice-based research networks: The laboratories of primary care research. Med Care 42(4 Suppl):III45–III49. Linde K, Hondras M, Vickers A, ter Riet G, Melchart D. 2001a. Systematic reviews of complementary therapies—An annotated bibliography. Part 3: Homeopathy. BMC Complemen Altern Med 1(1):4 (electronic publication). Linde K, Jonas WB, Melchart D, Willich S. 2001b. The methodological quality of randomized controlled trials of homeopathy, herbal medicines and acupuncture. Int J Epidemiol 30(3):526–531. Linde K, ter Riet G, Hondras M, Vickers A, Saller R, Melchart D. 2001c. Systematic reviews of complementary therapies—An annotated bibliography. Part 2: Herbal medicine. BMC Complement Altern Med 1(1):5 (electronic publication). Linde K, Vickers A, Hondras M, ter Riet G, Thormahlen J, Berman B, Yelchart D. 2001d. Systematic reviews of complementary therapies—An annotated bibliography. Part 1: Acupuncture. BMC Complement Altern Med 2001;1(1):3 (electronic publication). Linde K, Mulrow CD. St John’s wort for depression. Cochrane Database Syst Rev 2004; (1). Little CV, Parsons T. Herbal therapy for treating osteoarthritis. Cochrane Database Syst Rev 2004; (1). Lorig KR, Sobel DS, Stewart AL, Brown BW Jr, Bandura A, Ritter P, Gonzalez VM, Laurent DD, Holman HR. 1999. Evidence suggesting that a chronic disease self-management program can improve health status while reducing hospitalization: A randomized trial. Med Care 37(1):5–14. Lumley J, Watson L, Watson M, Bower C. Periconceptional supplementation with folate and/ or multivitamins for preventing neural tube defects. Cochrane Database Syst Rev 2004; (1). MacPherson H, White A, Cummings M, Jobst K, Rose K, Niemtzow R. 2001. Standards for reporting interventions in controlled trials of acupuncture: The STRICTA recommendations. Complement Ther Med 9(4):246–249. Mahomed K. Folate supplementation in pregnancy. Cochrane Database Syst Rev 2004; (1). Mahomed K. Iron and folate supplementation in pregnancy. Cochrane Database Syst Rev 2004; (1). Mahomed K. Iron supplementation in pregnancy. Cochrane Database Syst Rev 2004; (1). Manheimer E, Anderson D. 2002. Survey of public information about ongoing clinical trials funded by industry: Evaluation of completeness and accessibility. BMJ 325(7363):528–531. McAlister FA, Laupacis A, Wells GA, Sackett DL. 1999. Users’ guides to the medical literature: XIX. Applying clinical trial results B. Guidelines for determining whether a drug is exerting (more than) a class effect. JAMA 282(14):1371–1377. Melchart D, Linde K, Fischer P, Berman B, White A, Vickers A, Allais G. Acupuncture for idiopathic headache. Cochrane Database Syst Rev 2004; (1).
OCR for page 166
Complementary and Alternative Medicine in the United States Melchart D, Linde K, Fischer P, Kaesmayr J. Echinacea for preventing and treating the common cold. Cochrane Database Syst Rev 2004; (1). Moher D. 2002. The CONSORT guidelines: Improving the quality of research. Consolidated Standards of Reporting Trials. Interview by Bonnie Horrigan. Altern Ther Health Med 8(3):103–108. Moher D, Soeken K, Sampson M, Ben-Porat L, Berman B. 2002. Assessing the quality of reports of systematic reviews in pediatric complementary and alternative medicine. BMC Pediatr 2(1):3. National Institutes of Health. 1998. Acupuncture. NIH Consensus Conference. JAMA 280(17):1518–1524. National Library of Medicine. 2002. MEDLINE Fact Sheet. [Online]. Available: http://www.nlm.nih.gov/pubs/factsheets/medline.html [accessed June 16, 2004]. NCCAM (National Center for Complementary and Alternative Medicine). 2004. Centers for Research on Complementary and Alternative Medicine (CRC) Program FY 2004 Research Priorities. [Online]. Available: http://nccam.nih.gov/research/priorities/index.htm [accessed June 16, 2004]. Nutting PA. 1996. Practice-based research networks: Building the infrastructure of primary care research. J Fam Pract 42(2):199–203. Nutting PA, Beasley JW, Werner JJ. 1999. Practice-based research networks answer primary care questions. JAMA 281(8):686–688. Ortiz Z, Shea B, Suarez Almazor M, Moher D, Wells G, Tugwell P. Folic acid and folinic acid for reducing side effects in patients receiving methotrexate for rheumatoid arthritis. Cochrane Database Syst Rev 2004; (1). Osiri M, Welch V, Brosseau L, Shea B, McGowan J, Tugwell P, Wells G. Transcutaneous electrical nerve stimulation for knee osteoarthritis. Cochrane Database Syst Rev 2004; (1). Pittler MH, Ernst E. Horse chestnut seed extract for chronic venous insufficiency. Cochrane Database Syst Rev 2004; (1). Pittler MH, Ernst E. Kava extract for treating anxiety. Cochrane Database Syst Rev 2004; (1). Pollo A, Amanzio M, Arslanian A, Casadio C, Maggi G, Benedetti F. 2001. Response expectancies in placebo analgesia and their clinical relevance. Pain 93(1):77–84. Proctor ML, Hing W, Johnson TC, Murphy PA. Spinal manipulation for primary and secondary dysmenorrhoea. Cochrane Database Syst Rev 2004; (1). Proctor ML, Smith CA, Farquhar CM, Stones RW. Transcutaneous electrical nerve stimulation and acupuncture for primary dysmenorrhoea. Cochrane Database Syst Rev 2004; (1). Schulz KF, Chalmers I, Hayes RJ, Altman DG. 1995. Empirical evidence of bias. Dimensions of methodological quality associated with estimates of treatment effects in controlled trials. JAMA 273(5):408–412. Shea B, Wells G, Granney A, Zytaruk N, Robinson V, Griffen L, Hamel C, Ortiz Z, Peterson J, Adachi J, Tugwell P, Guyatt G. Calcium supplementation on bone loss in post-menopausal women. Cochrane Database Syst Rev 2004; (1). Shekelle P, Eccles MP, Grimshaw JM, Woolf SH. 2001. When should clinical guidelines be updated? BMJ 323(7305):155–157. Shiffman RN, Shekelle P, Overhage JM, Slutsky J, Grimshaw J, Deshpande AM. 2003. Standardized reporting of clinical practice guidelines: A proposal from the Conference on Guideline Standardization. Ann Intern Med 139(6):493–498. Silagy CA, Stead LF, Lancaster T. 2001. Use of systematic reviews in clinical practice guidelines: Case study of smoking cessation. BMJ 323(7317):833–836. Stange KC, Zyzanski SJ, Jaen CR, Callahan EJ, Kelly RB, Gillanders WR, Shank JC, Chao J, Medalie JH, Miller WL, Crabtree BF, Flocke SA, Gilchrest VJ, Langa DM, Goodwin MA. 1998. Illuminating the “black box”. A description of 4454 patient visits to 138 family physicians. J Fam Pract 46:377–389.
OCR for page 167
Complementary and Alternative Medicine in the United States Suresh GK, Davis JM, Soll RF. Superoxide dismutase for preventing chronic lung disease in mechanically ventilated preterm infants. Cochrane Database Syst Rev 2004; (1). Tabet N, Birks J, Grimley Evans J. Vitamin E for Alzheimer’s disease. Cochrane Database Syst Rev 2004; (1). Tang JL, Wong TW. 1998. The need to evaluate the clinical effectiveness of traditional Chinese medicine. Hong Kong Med J 4(2):208–210. Taylor JA, Weber W, Standish L, Quinn H, Goesling J, McGann M, Calabrese C. 2003. Efficacy and safety of echinacea in treating upper respiratory tract infections in children: A randomized controlled trial. JAMA 290(21):2824–2830. Taylor MJ, Carney S, Geddes J, Goodwin G. Folate for depressive disorders. Cochrane Database Syst Rev 2004; (1). ter Riet G, de Craen AJ, de Boer A, Kessels AG. 1998. Is placebo analgesia mediated by endogenous opioids? A systematic review. Pain 76(3):273–275. Thacker SB. 1992. Les principes et la pratique de la surveillance en santé publique; L’utilization des données en santé publique [The principles and practice of surveillance in public health; The use of data in public health]. Santé Publique 1992:43–44. Thacker SB, Berkelman RL. 1988. Public health surveillance in the United States. Epidemiol Rev 10:164–190. Thomas P, Griffiths F, Kai J, O’Dwyer A. 2001. Networks for research in primary health care. BMJ 322(7286):588–590. Towheed TE, Anastassiades TP, Shea B, Houpt J, Welch V, Hochberg MC. Glucosamine therapy for treating osteoarthritis. Cochrane Database Syst Rev 2004; (1). Vickers A. 1998. Bibliometric analysis of randomized trials in complementary medicine. Complement Ther Med 6:185–189. White AR, Ernst E. 2000. Economic analysis of complementary medicine: A systematic review. Complement Ther Med 8(2):111–118. Wilson ML, Murphy PA. Herbal and dietary therapies for primary and secondary dysmenorrhoea. Cochrane Database Syst Rev 2004; (1). Wilt T, Ishani A, MacDonald R, Rutks I, Stark G. Pygeum africanum for benign prostatic hyperplasia. Cochrane Database Syst Rev 2004; (1). Wilt T, Ishani A, MacDonald R, Stark G, Mulrow C, Lau J. Beta-sitosterols for benign prostatic hyperplasia. Cochrane Database Syst Rev 2004; (1). Wilt T, Ishani A, Stark G, MacDonald R, Mulrow C, Lau J. Serenoa repens for benign prostatic hyperplasia. Cochrane Database Syst Rev 2004; (1). Wilt T, MacDonald R, Ishani A, Rutks I, Stark G. Cernilton for benign prostatic hyperplasia. Cochrane Database Syst Rev 2004; (1). Zarin D, Pincus HA, West J, McIntyre JS. 1997. Practice-based research in psychiatry. Am J Psychiatry 154(9):1199–1208.
Representative terms from entire chapter: