National Academies Press: OpenBook

Complementary and Alternative Medicine in the United States (2005)

Chapter: 5 State of Emerging Evidence on CAM

« Previous: 4 Need for Innovative Designs in Research on CAM and Conventional Medicine
Suggested Citation:"5 State of Emerging Evidence on CAM." Institute of Medicine. 2005. Complementary and Alternative Medicine in the United States. Washington, DC: The National Academies Press. doi: 10.17226/11182.
×

5
State of Emerging Evidence on CAM

For policy makers, practitioners, patients, and health care system managers to make informed decisions about the use of complementary and alternative (CAM) therapies, they must have both access to and a means of evaluating the results of research on the topic. This chapter discusses the evidence that forms the basis for such decision making and the methods of evaluation, as well as the available resources providing access to the results of existing research on CAM interventions.

In CAM as well as in conventional medicine, randomized controlled trials (RCTs), when possible, are the preferable study design for assessing efficacy. RCTs use random allocation to create comparable groups, after which an intervention is introduced. This intervention consists of treatment for the test group and a placebo, no treatment, or another active treatment for the control group. Once the outcome is recorded, any observable differences between the treatment and control group should be attributable to the intervention because the groups were initially comparable before the intervention was introduced.

A systematic review uses explicit, systematic methods to review existing research, particularly the effectiveness of health care interventions, as evaluated by RCTs. Some systematic reviews may include meta-analyses, which provide an overview of the results of similar studies by the use of statistical methods to evaluate the data from many studies. Systematic reviews are widely considered the best method for gathering and synthesizing evidence as well as for determining gaps that exist in current research. Although basic science research and evaluation of cost-effectiveness are also important aspects of research on CAM therapies and modalities, the focus of the

Suggested Citation:"5 State of Emerging Evidence on CAM." Institute of Medicine. 2005. Complementary and Alternative Medicine in the United States. Washington, DC: The National Academies Press. doi: 10.17226/11182.
×

following sections is the evaluation of the clinical efficacies of therapies by RCTs and systematic reviews.

SOURCES OF INFORMATION ON HIGH-QUALITY EVIDENCE

Two main sources of information about published RCTs and systematic reviews are The Cochrane Library and MEDLINE. Critical reviews of reviews and Agency for Healthcare Research and Quality (AHRQ) Evidence Reports summarize the information by using rigorous and objective methods. National Institutes of Health (NIH) Consensus Statements incorporate evidence from RCTs and systematic reviews together with the judgments of a panel of nonadvocate, nongovernmental experts, to reach a decision about the efficacy and safety of a particular treatment.

MEDLINE

MEDLINE, a product of the National Library of Medicine, is an extensive bibliographic database covering all areas of clinical medicine and biomedical research. The bibliographic citations and abstracts indexed in MEDLINE are from more than 4,600 biomedical journals published worldwide, and the database includes information on all randomized trials in MEDLINE-indexed journals, regardless of the methodological quality or clinical relevance. MEDLINE is accessible online and is free of charge to the public, and most of its 12 million citations are available in English, at least as abstracts. The database includes studies published since 1966, the year that MEDLINE began, and is updated on a regular basis (National Library of Medicine, 2002).

In recent years, relevant indexing terms have been introduced on MEDLINE to facilitate queries on trials and systematic reviews related to CAM. MEDLINE introduced the publication type “randomized controlled trial” for specific RCTs in 1991 and the subject subset “systematic review” in 2001. The subject subset “CAM” was introduced in 2001 and includes all records identified through the execution of a complex, highly sensitive search strategy designed to identify all records in the MEDLINE database related to CAM. The introduction of these terms allows interested individuals to make simple queries of MEDLINE to estimate changes in the evidence base for CAM from the results of RCTs and systematic reviews over time. Figure 5-1 charts the tremendous growth in the number of RCTs over the past 20 years, and Figure 5-2 shows that the rate of increase of reviews and meta-analyses is even greater. These increases parallel general trends of growth in trials and meta-analyses over the past twenty years (Lee et al., 2001). Despite these developments, however, limitations of MEDLINE persist: not all studies in MEDLINE are indexed with the appropriate terms

Suggested Citation:"5 State of Emerging Evidence on CAM." Institute of Medicine. 2005. Complementary and Alternative Medicine in the United States. Washington, DC: The National Academies Press. doi: 10.17226/11182.
×

FIGURE 5-1 Number of CAM RCTs indexed on MEDLINE, 1982 to 2002. This search was performed on December 11, 2003, by using a search strategy with the following terms to obtain counts for each year: randomized controlled trial (publication type) AND year (publication date).

FIGURE 5-2 Number of reviews and meta-analyses related to CAM indexed on MEDLINE, 1982 to 2002. This search was performed on December 11, 2003, by using a search strategy with the following two sets of terms to obtain counts for each year: systematic (subset type) AND year (publication date) and meta-analysis (publication type) AND year (publication date).

Suggested Citation:"5 State of Emerging Evidence on CAM." Institute of Medicine. 2005. Complementary and Alternative Medicine in the United States. Washington, DC: The National Academies Press. doi: 10.17226/11182.
×

(Dickersin et al., 1994), and many reports, especially in the field of CAM, are not included on MEDLINE (Egger et al., 2003).

The Cochrane Library

The Cochrane Library, unique both for its scope and for its methodological standards, is supported through the work of the Cochrane Collaboration (Dickersin and Manheimer, 1998), an international organization of more than 9,000 contributors (mostly volunteers) from more than 80 countries (Allen and Clarke, 2003). The Cochrane Complementary Medicine Field, based at the University of Maryland Center for Integrative Medicine, coordinates all of the CAM-related activities of the Cochrane Collaboration, including the development of a database with information on more than 7,000 controlled trials of CAM therapies and modalities, and facilitates the preparation of CAM reviews and the promotion of these reviews, especially among the members of the CAM community.

Members of the Cochrane Collaboration prepare up-to-date, reliable summaries or systematic reviews of every kind of health care therapy. Cochrane reviews, which are intended to answer questions about health care and to guide providers in practical decision making about treatment, are published quarterly in The Cochrane Library. Although reviews of journal articles are current only as of their date of publication, the electronic format of The Cochrane Library allows the reviews to be updated easily and periodically to account for new evidence. In addition, Cochrane reviews have shown greater methodological rigor than systematic reviews and meta-analyses published in paper-based journals (Jadad et al., 1998, 2000). The use of rigorous methods is also ensured by the requirement that the protocols used for a review be prepared before a review is conducted and by an extensive system of peer review.

Because the information in The Cochrane Library is prescreened to a certain extent and includes only studies evaluating health care therapies, it is of generally higher quality and greater relevance to patient care than the information available on MEDLINE. The Cochrane Database of Systematic Reviews and the Cochrane Central Register of Controlled Trials comprise the major databases of the Cochrane Collaboration. If no review is available on the Cochrane Database of Systematic Reviews, one can check The Cochrane Library’s Database of Abstracts of Reviews of Effectiveness (a database of reviews collected by the Cochrane Collaboration but not prepared according to the strict standards of Cochrane reviews) or the applicable trials registered with the Cochrane Central Register of Controlled Trials.

The Cochrane database of CAM-related clinical trials is an immensely valuable resource because it covers many of the controlled trials that would

Suggested Citation:"5 State of Emerging Evidence on CAM." Institute of Medicine. 2005. Complementary and Alternative Medicine in the United States. Washington, DC: The National Academies Press. doi: 10.17226/11182.
×

not be included on MEDLINE. Vickers (1998) analyzed the Cochrane register for studies related to CAM and found that 19 percent of the citations were not indexed on MEDLINE. The trials included in the register were derived from 965 different journals; 84 percent of the trials were published in conventional medical journals. The numbers of trials per therapy varied a great deal: although Vickers found 554 trials of acupuncture and 804 of herbal medicine, he retrieved only 47 trials of aromatherapy. The number of trials per condition also varied, with 501 trials of cardiovascular disease, 386 trials of musculoskeletal disorders, and 293 trials of surgery-related symptoms, but only 11 trials of fatigue disorders. The objective of the register is to include all large multicenter trials, such as those recently published showing that St. John’s wort and echinacea are ineffective for the treatment of major depression and the common cold, respectively (Hypericum Depression Trial Study Group, 2002; Taylor et al., 2003). Also included are smaller RCTs, such as pilot studies of acupuncture conducted in China. The ultimate aim of developing the Cochrane CAM register is to provide a comprehensive source of trials of CAM therapies and modalities, thus reducing the need for systematic reviewers and others to search multiple sources.

At present The Cochrane Library contains 145 CAM-related systematic and an additional 340 non-Cochrane CAM-related systematic reviews (see Table 5-1 for a sampling of therapies covered by Cochrane and non-Cochrane reviews). These reviews cover many areas of CAM, with particular strength in the fields of acupuncture and herbal medicine, reflecting not only the large number of trials in these fields, but also the great interest of clinicians, policy makers, and consumers in these areas.

There are some disparities between evidence from Cochrane reviews and evidence from clinical practice. For example, although relaxation techniques (e.g., meditation) are the most commonly used CAM therapy among the U.S. general population (Eisenberg et al., 1998) and the fourth most commonly used therapy in hospital-based CAM or wellness centers (Health Forum, 2003), few Cochrane reviews have evaluated such therapies. On the other hand, although herbal therapy and treatment with other dietary supplements are not widely offered in U.S. hospitals, they are the most reviewed and are among the therapies that are the most commonly used by the U.S. public (Eisenberg et al., 1998).

The international structure of the Cochrane Collaboration plays a critical role in the identification of CAM trials and the preparation of reviews of CAM treatments and modalities because the therapies considered CAM in the United States are often the traditional medicines used by the populations of other countries. Through the work of the 14 Cochrane Centers worldwide, journals published around the world are hand searched to identify RCTs on conventional medicine therapies and CAM therapies and

Suggested Citation:"5 State of Emerging Evidence on CAM." Institute of Medicine. 2005. Complementary and Alternative Medicine in the United States. Washington, DC: The National Academies Press. doi: 10.17226/11182.
×

TABLE 5-1 Number of Cochrane and Non-Cochrane Reviews, by Therapy, March 2004

Therapy

Cochrane Reviews

Non-Cochrane Reviews

Acupuncture

10

69

Alexander technique

1

0

Art therapy

1

1

Biofeedback

2

26

Chiropractic

2

33

Dietary supplements (nonherbal)

71

46

Electromagnetic therapy

3

11

Herbal therapy

23

79

Homeopathy

4

34

Laser therapy

4

4

Massage therapy

4

18

Prayer

1

2

Transcutaneous electrical nerve stimulation

7

11

Therapeutic touch

1

3

Yoga

2

4

Other

27

91

NOTE: Some reviews cover multiple therapies and are therefore counted multiple times. The total number of Cochrane and Non-Cochrane reviews represented in this table are 156 and 340, respectively.

modalities that may be relevant and eligible for a systematic review. The Chinese Cochrane Centre, for example, has identified an estimated 10,000 trials of traditional Chinese medicine through their hand searches (Tang and Wong, 1998); moreover, dozens of reviews of traditional Chinese medicine are under way.

Cochrane Review Evidence for CAM1

All Cochrane reviews, be they of CAM or conventional medicine therapies, apply the same standards, that is, therapies within both categories are

1  

The committee did not include information about the general direction of effect for the AHRQ reports because the individual reports covered too wide a range of conditions (e.g., S-Adenosyl-L-Methionine (SAMe) for Depression, Osteoarthritis, and Liver Disease) and therapies (e.g., Mind-Body Interventions for Gastrointestinal Conditions). Cochrane reviews, in contrast, typically evaluate a specific therapy for a specific condition. Concise summaries of the findings of each of the AHRQ Technology Reports are presented on the AHRQ website or http://www.ahrq.gov/clinic/epcindex.htm#complementary.

Suggested Citation:"5 State of Emerging Evidence on CAM." Institute of Medicine. 2005. Complementary and Alternative Medicine in the United States. Washington, DC: The National Academies Press. doi: 10.17226/11182.
×

evaluated according to the strength of evidence from RCTs. To evaluate the evidence for CAM from Cochrane reviews, all reviews of CAM-related therapies were selected from The Cochrane Library and assigned categories, as described below. As a means of applying an objective, reproducible, and operational eligibility criterion, the committee considered Cochrane reviews to be related to CAM only if the therapies reviewed were listed as therapies in the National Center for Complementary and Alternative Medicine (NCCAM)-National Library of Medicine CAM on PubMed, a database of abstracts and articles on CAM-related therapies. The database can be accessed by use of a multipage search strategy designed to identify all studies listed on PubMed that should be indexed in the PubMed CAM subset. The results from all eligible Cochrane reviews of CAM therapies were assigned to one of the following six categories by two trained methodologists: positive effect, possibly positive effect, two active treatments are equal, insufficient or inconclusive evidence of an effect, no effect, or harmful effect. When the two raters differed on their classification of the treatment described in a review, a third rater trained in RCT and systematic review methodologies assigned the final classification. This rating system was used in a previous study to assess the evidence base for conventional medicine according to the information found in Cochrane reviews (Ezzo et al., 2001).

The agreement of the classification assignment between the initial two raters was 83 percent. For the 17 percent of reviews for which the initial raters assigned different classification codes, the third rater agreed with one of the initial two raters’ codes in all cases. The largest number of treatments described in the reviews were classified as insufficient evidence of an effect (n = 82; 56.6 percent), followed by positive effect (n = 36; 24.8 percent) and possibly positive effect (n = 18; 12.4 percent). Only one review described a treatment that was classified as harmful (Caraballoso et al., 2003) (see Table 5-2). The reviews describing treatments classified as having positive effects are listed in Table 5-3. Although this exercise suggests that there is strong evidence for the effectiveness of some CAM therapies, much more research is required, as demonstrated by the large proportion of reviews of treatments classified as insufficient evidence of an effect. The fact that only one of the treatments in the Cochrane reviews fell into the harmful effect category suggests that clinical trials of CAM therapies have posed little risk to the participants.

Some interesting findings emerge when the results of the evaluation of Cochrane reviews of CAM therapies are compared with the results of the earlier study (Ezzo et al., 2001) evaluating Cochrane reviews of conventional therapies: insufficient evidence of an effect was determined for a larger proportion of CAM therapies (56.6 percent for CAM versus 21.3

Suggested Citation:"5 State of Emerging Evidence on CAM." Institute of Medicine. 2005. Complementary and Alternative Medicine in the United States. Washington, DC: The National Academies Press. doi: 10.17226/11182.
×

TABLE 5-2 Conclusion Categories, Definitions, and Proportions Classified by Readers for Cochrane CAM Reviews

Conclusion Category

Definition

Readers’ Consensual Rating for Included Reviews (n = 145) (%)

Positive effect

Treatment is more beneficial/effective than control for the primary outcome.

36 (24.8)

Possibly positive effect

Treatment may have a positive effect, but a major unresolved methodological issue, such as all studies being very low quality, or findings based on only one study, precludes making a definitive statement.

18 (12.4)

Two active treatments are equal

Two biologically active treatments, such as two antibiotics, are equally as effective for the condition being treated. This category to be used only when comparing two active treatments, not placebo or no treatment.

1 (.6)

Insufficient/ inconclusive evidence

There is not sufficient evidence to determine effectiveness.

82 (56.6)

No effect

There is sufficient evidence, and there is no effect.

7 (4.8)

Harmful effect

Treatment does more harm than good.

1 (.6)

percent for conventional medicine), CAM therapies were less likely to be classified as harmful (8.1 percent for conventional medicine versus 0.69 percent for CAM) or as having no effect (20.0 percent for conventional medicine versus 4.8 percent for CAM), and classification of the therapies as having positive or a possibly positive effect was approximately equal for CAM and conventional medicine therapies (41.3 percent for conventional medicine versus 38.4 percent for CAM). In making comparisons between the two studies, however, it is important to keep in mind that the studies were conducted at different times and thus included different sets of Cochrane reviews. The study of Ezzo et al. (2001) included only those reviews published in The Cochrane Library at the time of its first issue in 1998, whereas the analysis of CAM described above included reviews published in the 2004 issue of the database, which is now much more comprehensive and better developed.

Suggested Citation:"5 State of Emerging Evidence on CAM." Institute of Medicine. 2005. Complementary and Alternative Medicine in the United States. Washington, DC: The National Academies Press. doi: 10.17226/11182.
×

TABLE 5-3 Cochrane CAM Reviewsa with Positive Effects

Study

Therapy

Indicationb

Limitation/ Commentc

Atallah AN et al., 2004

Calcium

Preventing hypertensive disorders and related problems in pregnancy

1, 4

Beckles WN et al., 2004

Omega-3 fatty acids (from fish oil)

Cystic fibrosis

1, 2

Brosseau L et al., 2004

Transcutaneous electrical nerve stimulation (TENS)

Rheumatoid arthritis in the hand

1

D’Souza RM and D’Souza R, 2004

Vitamin A

Measles

4

Darlow B and Austin N, 2004

Selenium

Short-term morbidity in preterm neonates

4

Darlow BA and Graham PJ, 2004

Vitamin A

Preventing morbidity and mortality in very low-birthweight infants

1, 4

Douglas RM et al., 2004

Vitamin C

Preventing and treating the common cold

1 (does not prevent colds; only reduces duration of symptoms)

Evans JR, 2004

Antioxidant vitamin and mineral supplements

Age-related macular degeneration

4, 5

Farmer A et al., 2004

Fish oil

Type 2 diabetes mellitus

3, 5

Furlan AD et al., 2004

Massage

Low back pain

2, 6

Herxheimer A and Petrie KJ, 2004

Melatonin

Preventing and treating jet lag

5

Suggested Citation:"5 State of Emerging Evidence on CAM." Institute of Medicine. 2005. Complementary and Alternative Medicine in the United States. Washington, DC: The National Academies Press. doi: 10.17226/11182.
×

Study

Therapy

Indicationb

Limitation/ Commentc

Homik J et al., 2004

Calcium and vitamin D

Corticosteroid-induced osteoporosis

Howlett A and Ohlsson A, 2004

Inositol

Respiratory distress syndrome in preterm infants

Hulme J et al., 2004

Electromagnetic fields

Osteoarthritis

Jepson RG et al., 2004

Cranberries

Preventing urinary tract infections

1

Linde K and Mulrow CD, 2004

St. John’s wort

Depression

2

Little CV and Parsons T, 2004

Herbal therapy

Osteoarthritis

—(only one herb found effective)

Lumley J et al., 2004

Periconceptual supplementation with folate and/or multivitamins

Preventing neural tube defects

4

Mahomed K, 2004

Folate supplementation in pregnancy

Biochemical parameters and pregnancy outcome

3, 4

Mahomed K, 2004

Iron and folate supplementation in pregnancy

Biochemical parameters and pregnancy outcome

3, 4

Mahomed K, 2004

Iron supplementation in pregnancy

Biochemical parameters and pregnancy outcome

3, 4

Melchart D et al., 2004

Acupuncture

Headache

Melchart D et al., 2004

Echinacea

Preventing and treating the common cold

1

Ortiz Z et al., 2004

Folic acid and folinic acid

Reducing side effects in patients receiving methotrexate

1

Suggested Citation:"5 State of Emerging Evidence on CAM." Institute of Medicine. 2005. Complementary and Alternative Medicine in the United States. Washington, DC: The National Academies Press. doi: 10.17226/11182.
×

Study

Therapy

Indicationb

Limitation/ Commentc

Osiri M et al., 2004

TENS

Knee osteoarthritis

1

Pittler MH and Ernst E, 2004

Horse chestnut

Chronic venous insufficiency

Pittler MH and Ernst E, 2004

Kava

Anxiety

5

Proctor ML et al., 2004

TENS and acupuncture

Primary dysmenorrhoea

1

Shea B et al., 2004

Calcium

Bone loss

Taylor MJ et al., 2004

Folate

Depression

4

Towheed TE et al., 2004

Glucosamine

Osteoarthritis

1, 2

Wilson ML and Murphy PA, 2004

Herbal and dietary therapies

Primary and secondary dysmenorrhoea

Wilt T et al., 2004

African prune

Benign prostatic hyperplasia

1, 2

Wilt T et al., 2004

Beta-sitosterols

Benign prostatic hyperplasia

2

Wilt T et al., 2004

Saw palmetto

Benign prostatic hyperplasia

2

Wilt T et al., 2004

Cernilton

Benign prostatic hyperplasia

1, 2, 6

aOur application of the list of terms used in the CAM on PubMed search strategy in defining our eligibility criteria for CAM-related resulted in the inclusion in our sample of some Cochrane reviews that many would not consider to be CAM-related. It is notoriously difficult to make the determination of whether or not a therapy should be considered CAM-related, and this decision must often be made in the context of the therapy’s application (e.g., for nutrients, whether it is used as a supplement or to address a deficiency); the setting in which the therapy is used (e.g., hospital, self-care); the current state of the evidence for the therapy (e.g., a therapy such as folic acid for neural tube defects has strong supportive evidence which has resulted in its integrated into the health care system); and the point of historical time at which the evaluation of the therapy as CAM or not CAM is made. Our CAM eligibility criteria, while initially deemed the most objective and reproducible approach for selecting CAM reviews, does not take into account the indication for which the therapy is used and has resulted in our including multiple reviews that are not CAM (e.g., vitamin A for measles) and excluding other reviews that are CAM (e.g., speleotherapy for the treatment of asthma), but the therapy reviewed is not a term in the CAM on PubMed search strategy. Because indexing reports always involves some degree of subjectivity (especially when indexing on a difficult-to-pin-down term such as CAM) and because the CAM on PubMed search strategy still requires improvements in the recall and precision of its terms, a second review, by an authority in the field, of reviews that were possibly inappropriately included or excluded based on the strict application of the CAM on PubMed search strategy terms, may be necessary.

Suggested Citation:"5 State of Emerging Evidence on CAM." Institute of Medicine. 2005. Complementary and Alternative Medicine in the United States. Washington, DC: The National Academies Press. doi: 10.17226/11182.
×

Study

Therapy

Indicationb

Limitation/ Commentc

bIndication is for the treatment of the condition unless specifically listed as prevention

c1= Optimal dosage, preparation, or method of administration needs further investigation

2= Long-term effectiveness data lacking

3= Effects only demonstrated on blood marker surrogate outcome measures

4= Results may only apply to populations with low intake of nutrient

5= May be possible adverse effects in some people, according to either the review data or other research

6= These reviews found positive effects in comparison to both active and inactive controls; for all others, positive effects were in comparison with inactive controls

Sufficiently powered, well-designed RCTs of adequate duration may be necessary to confirm the positive conclusions of these Cochrane teviews.

Ongoing Trials

An often cited limitation of the systematic review is that it is generally restricted to published RCTs; if the published RCTs are not representative of all RCTs undertaken (i.e., both published and unpublished RCTs), the review may be unreliable (Manheimer and Anderson, 2002). Although various groups have tried to identify, organize, and disseminate information on unpublished and ongoing RCTs, much work remains to be done. One attempt to bridge the information gap is the website Current Controlled Trials (http://www.controlled-trials.com), an international resource providing metadata about registers of controlled trials in a searchable database. Government agencies also compile registers: the NCCAM website contains a list of NCCAM-funded clinical trials, whereas ClinicalTrials. gov provides access to a database of thousands of clinical studies being sponsored by NIH, other federal agencies, and the pharmaceutical industry. However, none of these resources is comprehensive at this time; only a comprehensive register of all ongoing CAM trials would allow researchers to know which CAM-related RCTs have been conducted, regardless of whether results are available.

Suggested Citation:"5 State of Emerging Evidence on CAM." Institute of Medicine. 2005. Complementary and Alternative Medicine in the United States. Washington, DC: The National Academies Press. doi: 10.17226/11182.
×

Summarizing the Reviews

In recent years, many systematic reviews of CAM therapies have been conducted and efforts are under way to synthesize and summarize as comprehensively as possible the evidence available from these reviews.

Systematic Reviews of Reviews

One way in which data are synthesized is through systematic reviews of reviews, which provide an overview of a therapy’s effectiveness across all conditions. It is important to note that those preparing the summary must be aware of and acknowledge or adjust for the fact that some studies may appear in more than one review of a given topic.

Reviews of reviews use comprehensive searches, strict inclusion criteria, and data extraction with pretested forms. These reviews of reviews summarize the basic information from individual reviews, including conditions, interventions, methodological features, and results, as well as present the number of studies reviewed and the reviewers’ own conclusions (Linde et al., 2001a,c,d).

AHRQ Evidence Reports

AHRQ, the leading federal agency concerned with research on health care quality, efficiency, effectiveness, and safety, prepares evidence reports and technology assessments that provide information on the clinical efficacies of medical interventions on the basis of systematic reviews and, when appropriate, meta-analyses. Many AHRQ evidence reports relate to CAM interventions (Table 5-4).

NIH Consensus Statement

An NIH consensus statement is prepared by a panel of experts who review key questions and data about various therapies before an audience that comprises other experts in various medical fields. The panel, working with this audience of experts, addresses a predefined question and reaches conclusions on the basis of both the scientific data presented to the panel and data from the relevant literature gathered from MEDLINE. In 1997, NIH produced a consensus statement on acupuncture and concluded that, despite equivocal results in many studies, acupuncture is clearly effective for postoperative dental care and the prevention of nausea and vomiting in adults after surgery and chemotherapy and is possibly effective for many other conditions (National Institutes of Health, 1998). Nevertheless, poor study designs, inadequate sample sizes, and other problems invalidated the

Suggested Citation:"5 State of Emerging Evidence on CAM." Institute of Medicine. 2005. Complementary and Alternative Medicine in the United States. Washington, DC: The National Academies Press. doi: 10.17226/11182.
×

TABLE 5-4 CAM Evidence Reports and Technology Assessmentsa of the U.S. Agency for Health Care Research and Qualityb (as of January 2004)

Title

Year of Publication

Acupuncture for fibromyalgia

2003

Acupuncture for osteoarthritis

2003

Antioxidant supplements, prevention and treatment of cancer

2003

Antioxidant supplements, prevention and treatment of cardiovascular disease

2003

Ayurvedic interventions for diabetes mellitus: A systematic review

2001

Ephedra and ephedrine for weight loss and athletic performance enhancement: Clinical efficacy of and side effects

2003

Garlic: Effects on cardiovascular risks and disease, protective effects against cancer, and clinical adverse effects

2000

Milk thistle: Effects on liver disease and cirrhosis and clinical adverse effects

2000

Mind-body interventions for gastrointestinal conditions

2001

S-Adenosyl-L-Methionine (SAMe) for depression, osteoarthritis, and liver disease

2002

aEvidence Reports and Technology Assessments are based on a comprehensive review of the literature, together with rigorous qualitative, and as appropriate, quantitative methods of synthesizing data from multiple studies.

bFull versions of these reports, as well as brief summaries, are available through the AHRQ website (www.ahrq.gov).

results of many studies, as did problems with the inclusion of appropriate controls. The NIH consensus statement recommended further research, as future studies will probably discover additional therapeutic uses for acupuncture.

Although systematic reviews are not themselves meant to function as recommendations or guidelines, they should ideally form the basis for guideline creation. Governments and medical institutions may adopt various methods of summarizing reviews and translating the findings from those reviews into recommendations for use by clinicians (Grol and Grimshaw, 2003; Shekelle et al., 2001; Shiffman et al., 2003; Silagy et al., 2001). Such organizations must also evaluate the economic costs of CAM therapies, as

Suggested Citation:"5 State of Emerging Evidence on CAM." Institute of Medicine. 2005. Complementary and Alternative Medicine in the United States. Washington, DC: The National Academies Press. doi: 10.17226/11182.
×

well as their efficacies, when deciding whether or not to offer them to patients (Friedman et al., 1995; Lorig et al., 1999). A recent systematic review of economic analyses of CAM therapies showed that, for the most part, “there is a paucity of rigorous studies that could provide conclusive evidence of differences in costs and outcomes between CAM therapies and orthodox medicine” (White and Ernst, 2000).

Evaluating Study Quality

Importance of Quality

As the discussion above makes clear, a substantial base of information comprising the results of RCTs and systematic reviews evaluating the effectiveness of CAM therapies now exists. The quality of these studies varies however, and the lower-quality RCTs have exaggerated CAM treatment effects (Juni et al., 2001; Schulz et al., 1995). Thus, the evaluation of study quality is of utmost importance to determine the validity of study results. Quality evaluations, already well under way in conventional medicine, are important to ensure the validity and quality of the research and should therefore become standard practice in CAM as well. To address this important issue, researchers have begun devising methods for the optimal conduct of RCTs of CAM therapies, as well as for evaluation of the quality of the research already conducted.

Important Components of Quality

Because quality varies across studies and it was found in RCTs of conventional medicine therapies that lower-quality RCTs have exaggerated treatment effects (Juni et al., 2001; Schulz et al., 1995), it is important to establish standards for evaluation. Study quality can be evaluated by various strategies, but any instrument used to evaluate quality must take into account randomization, blinding, dropouts, and allocation concealment. For example, the Jadad Scale (Jadad et al., 1996) evaluates the quality of reporting by asking a variety of questions: Was the study described as randomized? Was the study described as double-blind? Was there a description of the study participants who withdrew and dropped out? These questions help to ascertain whether the conduct and reporting of the trial are adequate. It is also important that RCTs have a sample size large enough to determine the effectiveness of the intervention. Systematic reviews are evaluated on the basis of whether they undertake a comprehensive literature search with unambiguous inclusion and exclusion criteria for studies, as well as on the basis of whether they use explicit and transparent methods to evaluate and summarize study data.

Suggested Citation:"5 State of Emerging Evidence on CAM." Institute of Medicine. 2005. Complementary and Alternative Medicine in the United States. Washington, DC: The National Academies Press. doi: 10.17226/11182.
×
Methodological Quality of CAM-Related RCTs

The research conducted thus far tends to show not only that RCTs of CAM have significant shortcomings and omissions both in the methodology and in the reporting but also that the quality of CAM-related RCTs is inconsistent (Linde et al., 2001b). For example, larger trials included on MEDLINE and published in English are, in general, of higher quality than those harder-to-find trials not published in English (Egger et al., 2003; Linde et al., 2001a,c,d). Most trials of homeopathy, herbal medicine, and acupuncture had major problems with their reporting and the study methodology, such as the documentation of allocation concealment, dropouts, and withdrawals. These shortcomings also varied by intervention (Linde et al., 2001b).

Efforts to Improve Quality

Many efforts are under way to improve the quality of trials and systematic reviews, including the establishment of CONSORT, QUOROM, and STRICTA (Standards for Reporting Interventions in Controlled Trials of Acupunture) guidelines. The CONSORT statement helps to improve the quality of RCTs by providing a checklist and flow diagram against which the trial procedures can be measured, as well as by standardizing the ways in which the findings from such trials are reported. The CONSORT checklist includes 22 items that should be included in reports describing RCTs, and its accompanying flowchart outlines how patients move through the trial process. The CONSORT statement is published in several languages and has received the endorsement of several prominent medical journals, including The Lancet, Annals of Internal Medicine, and the Journal of the American Medical Association (JAMA). As of 2002, CONSORT was being used by about 500 journals, most leading editorial groups, and many granting agencies (Moher, 2002). Wider dissemination of the CONSORT guidelines within the CAM community (Moher, 2002) should improve the quality of reports of CAM-related RCTs in the future. A related document, the QUOROM statement, provides specific guidelines for the reporting of metaanalyses of RCTs, as well as a checklist and flow diagram to promote standardization and the inclusion of critical components. The CONSORT and QUOROM checklists and diagrams are available at http://www.consort-statement.org.

Similarly, STRICTA provides a set of guidelines based on the CONSORT statement and is meant to serve as an acupuncture-oriented supplement to the CONSORT guidelines. An international group of acupuncture practitioners and researchers established STRICTA guideliness to promote accurate and adequate reporting of acupuncture trials; editors of several

Suggested Citation:"5 State of Emerging Evidence on CAM." Institute of Medicine. 2005. Complementary and Alternative Medicine in the United States. Washington, DC: The National Academies Press. doi: 10.17226/11182.
×

journals have also contributed to the drafting of these guidelines. Participating journals publish the STRICTA guidelines and ask their prospective authors to adhere to the standards (MacPherson et al., 2001).

Although CAM therapies are often criticized for being used despite a lack of evidence, hundreds of systematic reviews have, in fact, evaluated specific CAM therapies; of these, some have been well conducted and have shown that the CAM therapy offers a clear benefit. Much more research is required however to demonstrate clearly whether other commonly used therapies are effective. Unfortunately, funding for CAM trials and systematic reviews is limited and generally must come from public resources. Private industry rarely funds research on CAM because the ability to patent natural products or CAM therapies is low. The evidence base for conventional medicine is also not complete, and as discussed above with respect to the study of Ezzo et al. (2001), systematic reviews demonstrate that only 22.5 percent of conventional medicine interventions have a clearly positive effect. Comparisons of quality of systematic reviews of CAM and conventional medicine therapies have shown that they share similar weaknesses (Moher et al., 2002). One of the most important omissions in systematic reviews was a lack of consistent reporting about the methods used to evaluate and assess the quality of the primary studies reviewed (Moher et al., 2002).

Twenty years ago very few RCTs of CAM therapies had been conducted, and the systematic review methodology was in its infancy. In the past twenty years, however, remarkable progress has been made in terms of the number of trials of CAM therapies conducted, improvements in trial quality, the development of systematic review methods for evaluating such trials, and the development of the infrastructure of the Cochrane Collaboration to support such rigorous evaluations. Although much work remains to be done, a great deal has been accomplished in a short time, from improvements to the indexing terms used and the creation of databases to the establishment of reporting guidelines and quality measurements.

As a result of these efforts, true evidence-based CAM is becoming a reality. With continued financial and institutional support, the success of this enterprise will continue into the future. Efforts to provide evidence-based CAM would be greatly facilitated by a system(s) that could be used to identify, within each agency of the federal government, completed and ongoing research on CAM therapies. Such a system would enable investigators to more quickly and accurately review what is known about various areas of CAM. The clinical trial registry sponsored by NIH and the Food and Drug Administration provides useful information on clinical trials; however, other research activities on CAM are not included in this registry. Therefore, the committee recommends that the federal government develop systems that can be used to identify CAM research and expenditures at the

Suggested Citation:"5 State of Emerging Evidence on CAM." Institute of Medicine. 2005. Complementary and Alternative Medicine in the United States. Washington, DC: The National Academies Press. doi: 10.17226/11182.
×

National Institutes of Health, the Centers for Disease Control and Prevention, the Health Resources Service Administration, Medicare, Medicaid, the U.S. Department of Agriculture, the U.S. Department of Defense, the U.S. Department of Veterans Affairs, and other federal agencies as appropriate.

The following section describes some of the gaps in evidence for CAM therapies and explores options that can be used to fill those gaps.

GAPS IN EVIDENCE

The scientific evidence that is developing in CAM research, reviewed earlier in this chapter, is primarily based on findings from clinical research on CAM treatments. This clinical research, which is itself still in a nascent phase and which is sometimes flawed, represents just one facet of the research that is needed. There are very large gaps in basic research on the underlying mechanisms through which CAM treatments affect outcomes, clinical research that compares CAM interventions with the interventions used in conventional medicine, and cost-effectiveness and health care utilization studies. Furthermore, as discussed in Chapter 4, advances in CAM are dependent on the ability to address complex methodological challenges created by the unique characteristics of CAM therapies.

Clinical Research

Although most research on CAM is clinical, there is a paucity of clinical research in which CAM interventions are compared with each other or with conventional medicine interventions. As discussed in Chapter 4, one of the obstacles to comparing CAM treatments with conventional medicine treatments is that CAM treatments often consider outcomes that are not ordinarily considered by conventional medicine research, such as “being centered” or “connectedness to the CAM provider or family members.” Conversely, conventional medicine studies often consider outcomes such as functional status and disease-specific outcomes that may not be used in studies of CAM therapies. Each type of medicine—conventional medicine and CAM—should make efforts to include the outcomes of the other type to facilitate valid comparisons between CAM and conventional medicine treatments. For example, measures of mood, quality of life, and preferences for outcome states that are used in clinical studies of conventional medicine therapies could be included in studies of CAM therapies. Measures of the outcomes that are often of interest to CAM patients and providers need to be developed that can then be used in conventional medicine research as well as CAM research. Collaboration between CAM researchers and social

Suggested Citation:"5 State of Emerging Evidence on CAM." Institute of Medicine. 2005. Complementary and Alternative Medicine in the United States. Washington, DC: The National Academies Press. doi: 10.17226/11182.
×

and behavioral scientists skilled in measurement development could be fruitful in this regard.

Basic Science

A great opportunity for scientific discovery in the basic science of CAM treatments is at hand. Of the basic research that has been done, botanicals have probably received the greatest amount of attention, in the form of studies of individual botanicals, botanical-drug interactions, and the identification of new drugs. NCCAM, like other institutes, centers, and offices of NIH, is giving a high priority to studies to determine the active ingredients, dosing, pharmacology, stability, and bioavailability of CAM therapies (NCCAM, 2004). Some of the formative research in this area has produced findings on the lack of identification of isoflavonone formononetin in a variety of black cohosh (Cimicifuga racemosa) populations. Isoflavonone formononetin is believed to be important in the reduction of menopausal vasomotor symptoms, and the findings of research of black cohosh given to menopausal women indicate that estrogen activity is due to compounds other than formononetin (Kennelly et al., 2002). Licorice root (Glycrrhiza glabra) induces apoptosis, G2/M cell cycle arrest, and Bcl-2 phosphorylation in tumor cell lines (Rafi et al., 2002), and pharmacokinetic studies of purified soy isoflavones show that high doses are rapidly eliminated by healthy males resulting in minimal toxicity (Busby et al., 2002). These are only a handful of the high-quality investigations on the basic science of medicinal plants that have been reported, while many others have been completed or are ongoing.

The area of mind-body medicine also offers exciting opportunities for basic research. Advances in technology enable research, for example, on the effects of mind-body techniques such as the effects of meditation on the brain, the endocrine system, and the immune system. New imaging techniques make it possible to study the phenomenon of acupuncture analgesia and placebo effects. For example, ter Riet et al. (1998) conducted a systematic review of six studies on the mechanism of placebo analgesia in humans and concluded that the studies provide evidence that a placebo analgesia effect exists. Pollo et al. (2001) examined whether expectations about treatment influence analgesic effects. They found that “different verbal instructions about certain and uncertain expectations of analgesia produce different placebo analgesic effects, which in turn trigger a dramatic change of behavior leading to a significant reduction of opioid intake”. These are but a few of many examples of areas of inquiry that hold great promise for understanding basic mechanisms of action relevant to the practice of CAM.

Suggested Citation:"5 State of Emerging Evidence on CAM." Institute of Medicine. 2005. Complementary and Alternative Medicine in the United States. Washington, DC: The National Academies Press. doi: 10.17226/11182.
×

Cost-Effectiveness Studies

Although a number of clinical studies are examining the efficacies of CAM treatments, very little research has tracked the costs associated with the treatment outcomes. Research on the cost-effectiveness of CAM therapies is hindered by a lack of consistency of treatments, a lack of standardized coding, and defects in clinical trials (pertinent to both conventional medicine and CAM treatment trials). These defects include the fact that the clinical trials are underpowered, a lack of an adequate description of subject recruitment, the use of suboptimal controls, and the use of single-center trials, which threatens generalizability and power.

Cross-Disciplinary Research

Research on CAM treatments benefits from the contributions of more than one discipline. In addition to providers who have specialized knowledge of CAM treatments and methodologists who can address the challenges inherent in CAM research design, CAM research can benefit by the inclusion of scientists with backgrounds in fields such as psychology, sociology, anthropology, economics, genetics, pharmacology, neuroscience, health services research, and health policy. These individuas can address the multiplicity of factors that influence those who use CAM treatments and the outcomes of those treatments.

Engaging experts from multiple fields in the investigation of CAM therapies provides an excellent opportunity to conduct transdisciplinary research. “Transdisciplinary research involves broadly constituted teams of researchers that work across disciplines in the development of the research questions to be addressed” (IOM, 2003b). Transdiciplinary research on CAM therapies could involve teams composed of a rheumatologist, immunologist, neuroscientist, epidemiologist, biostatistician, traditional Chinese medicine practitioner, and physician who is trained in conventional medicine and CAM and is investigating chronic pain disorders such as osteoarthritis.

Research Investigators

Established scientists are drawn to a new field by interesting questions, especially when such questions can be approached by using the scientists’ knowledge and skills and by the availability of resources. Relatively few established scientists are engaged in CAM research, however, and certain areas of CAM are ripe for research. Mind-body medicine provides a good example. Mind-body medicine concerns the effects of the brain on biophysiological processes and clinical outcomes. Clinical researchers and

Suggested Citation:"5 State of Emerging Evidence on CAM." Institute of Medicine. 2005. Complementary and Alternative Medicine in the United States. Washington, DC: The National Academies Press. doi: 10.17226/11182.
×

basic scientists who share interest in these questions have traditionally pursued their interests separately and have published their findings in different journals. However, the public’s growing interest in practices such as meditation as a way of reducing the risk for and recovery from illness is motivating a more clinical orientation by basic scientists and a more basic science orientation by clinical researchers.

Advances in the cognitive and affective neurosciences and technical advances in imaging techniques are fueling the convergence of clinical science and basic science. A study by Davidson et al. (2003) provides a good example of this convergence. These investigators conducted an RCT of the effects of a meditation-based clinical training program on brain and immune function. At the conclusion of the training, subjects in both groups were vaccinated with influenza vaccine. Meditators showed a significant increase in left-sided anterior activation compared with that for the nonmeditators, and they also had a significant increase in antibody titers compared with the titers in the nonmeditators.

Scientists in other areas should be made aware of the exciting questions about CAM that remain to be answered. Such interest can be fanned in a variety of ways, including conferences focused on particular CAM modalities that invite science-oriented CAM practitioners and scientists from other areas to participate. Invitations to scientists to apply their knowledge and skill to CAM research questions will be more attractive if they include certain kinds of collaborations. Both basic research and clinical research in CAM involve subject matter that is likely to be beyond the conventional scientist’s knowledge base and scientists new to CAM will often need to be partnered with individuals who are expert in the focal CAM modality. Furthermore, investigation of a number of CAM modalities, such as traditional Chinese medicine, by conventional scientific techniques is scientifically problematic and requires the collaboration of methodologists who can create appropriate and effective research designs (see Chapter 4).

The development of a workforce to conduct research on CAM will proceed more rapidly if critical masses of investigators can be created in various locations. Furthermore, CAM research should thrive in an interdisciplinary environment. Centers that bring together scientists from diverse disciplines can help create these interdisciplinary groups of scientists focused on particular aspects of CAM. The cross-collaborations and synergy that characterize a strong center will contribute to the more rapid growth of a cadre of scientists who will be able to advance the research in CAM.

As discussed in Chapter 1, one of the goals of the Academic Health Centers for Integrative Medicine is to “help transform medicine and health care through rigorous scientific studies” (see Appendix B for a list of centers). One such center at the University of Maryland has grown out of a partnership of funding from private foundations with university support

Suggested Citation:"5 State of Emerging Evidence on CAM." Institute of Medicine. 2005. Complementary and Alternative Medicine in the United States. Washington, DC: The National Academies Press. doi: 10.17226/11182.
×

and resources. This combination allowed the start of pilot research projects that could generate preliminary data that were used to make successful applications to NIH for grants for CAM research including a center grant. The University of Maryland model (which started in 1991) has relied on a collaborative team approach whereby clinicians experienced in CAM modalities work with specialists in rheumatology and pain (the particular focus of the center) as well as seasoned methodologists and statisticians to design trials that are sensitive to the unique practice of the CAM modality while meeting the highest methodological standards. The center grant funding has been crucial in establishing various infrastructural cores. For example, the first core, the database and evaluation core, locates the existing literature in the field, synthesizes the data, and conducts systematic reviews to aid in formulating appropriate research hypotheses and designing appropriate studies. Until recently this task was greatly hindered by the fact that CAM journals were not listed in MEDLINE, and literature search strategies such as those that use keywords were inadequate. However, this situation has changed, and many of the difficulties have improved.

A second core of the Maryland center is administrative and provides oversight and management for all stages of the research process. A statistical, epidemiological, and data management core, the third core, provides consultation on all aspects of study design, data collection, and statistical analysis. Studies funded by the center grant have completed the Phase II level of development, which has led to large, fully developed Phase III clinical trials, in addition to allowing transitional research from basic science studies through clinical trials. Furthermore, a program of development and feasibility studies and a training and education program have built on the concept of a collaborative model, encouraging innovative lines of investigation and the nurturing of new investigators in the field. These programs have allowed the center to bring together experienced investigators from a broad range of medical disciplines to work with the center team on pilot research studies, and it has also mentored junior faculty and postdoctoral fellows as they develop lines of inquiry and embark on independent careers in the field.

The field of CAM also needs a cadre of new junior researchers. Models of training programs in basic and clinical research for conventional medicine can be found in major U.S. health sciences campuses. The challenge is to induce such programs to include training in clinical research for CAM. One approach is to build upon existing research training programs by adding specific CAM content for the trainees in CAM. This could be done through postdoctoral training programs that are designed for basic and clinical research in conventional medicine. This is an efficient way to develop CAM researchers during the period when not enough CAM researchers are available to serve as faculty for a complete program. A

Suggested Citation:"5 State of Emerging Evidence on CAM." Institute of Medicine. 2005. Complementary and Alternative Medicine in the United States. Washington, DC: The National Academies Press. doi: 10.17226/11182.
×

second approach is to provide supplemental grants to existing grants that will fund a junior researcher to conduct CAM research and benefit from mentoring by the principal investigator of the existing grant. NIH has used this approach to help increase the numbers of researchers from underrepresented minorities. A third approach is to award individual career development grants that allow the junior researcher of CAM to be trained by established researchers from around the country. A fourth approach is to provide intensive workshops at a central location followed by mentoring and technical support for the participants when they return to their home institutions.

All the approaches described above involve programs for people with conventional M.D. or Ph.D. training. Chapter 8 discusses the training of CAM practitioners to conduct research. The next section explores a framework for research and the translation of validated therapies into practice.

A RESEARCH FRAMEWORK

During its deliberations and analyses, the committee concluded that research on CAM is inextricably linked to the practice of CAM therapies for many reasons, an important one of which is that CAM therapies are already in widespread use today and it is reasonable to attempt to evaluate the outcomes of that use. Another is that in the practice setting one can focus on research that answers questions about how therapies function in the real world where patients vary, often have multiple problems, and are using multiple therapies. Such a focus not only generates research that addresses real-world practice issues but also facilitates the adoption of practice changes on the basis of those research results (Green and Dovey, 2001; Nutting et al., 1999). Therefore, the committee believes that it is equally important both to continue to conduct research aimed at determining efficacy and uncovering mechanisms of action and to engage in research aimed at investigating what is occurring in practice. Furthermore, the gathering and analysis of accurate information about CAM (and conventional medicine) practices and their use are crucial to measuring and understanding outcomes of care.

Chapter 2 of this report discussed what is known about the use of CAM by the American public. That chapter also described how little information is available about

  • CAM use by specific populations;

  • how the American public gathers and evaluates information and makes decisions about accessing CAM therapies;

  • what motivates patients to use CAM;

  • adherence to CAM treatment or self-treatment with CAM;

Suggested Citation:"5 State of Emerging Evidence on CAM." Institute of Medicine. 2005. Complementary and Alternative Medicine in the United States. Washington, DC: The National Academies Press. doi: 10.17226/11182.
×
  • outcomes, including adverse events, of single CAM therapies, multiple CAM therapies, and CAM therapies in combination with conventional therapies; and

  • the extent to which CAM use is a trigger for positive behavioral change.

This chapter discusses what is known about the efficacy and effectiveness of CAM therapies and the need for additional basic science, cost-effectiveness, and cross-disciplinary research in CAM. To address these gaps, the committee has developed the research model illustrated in Figure 5-3. Four major components of this model are national surveys (both periodic and longitudinal surveys), a sentinel surveillance system, practice-based research networks (PBRNs), and CAM research centers.

National Surveys

The first component of the research model illustrated in Figure 5-3 is a set of interactive surveys. Data from these surveys are needed to capture CAM use by the American public in a timely manner for several distinct, but interrelated purposes. First, there is a need for periodic (e.g., every 5 years), large, comprehensive surveys assessing the prevalence, patterns, perceptions, and costs of CAM therapy use by a representative national survey of adults living in the United States. This will provide data on national trends on CAM use that will enable comparisons over time by class of patient, e.g., by ethnicity, gender, area of the country, insurance status, and the general rate of use of CAM. In designing such surveys, oversampling of various ethnic minority populations is necessary to ensure the collection of high-quality data that document the differences in the patterns of CAM use by both preference and level of access to all types of health care services among minority (and majority) populations.

Second, questions related to CAM should be included on annual and semiannual federally funded surveys focusing on the health care of the American public. The recently completed study by the National Center for Health Statistics is an excellent example of the type of survey needed (Barnes et al., 2004). It is also important to establish a set of common questions that can be rotated (because of time constraints) to enable the more frequent tracking of a core set of key variables (e.g., prevalence of CAM use, major disease categories treated by CAM, average numbers of visits by CAM users, and out-of-pocket expenditures).

Third, questions related to CAM should be included on ongoing, longitudinal, cohort studies to enable documentation of the patterns of CAM use over time and their relationship (or lack thereof) to health outcomes. Longitudinal cohort studies have produced data that result in major benefits to

Suggested Citation:"5 State of Emerging Evidence on CAM." Institute of Medicine. 2005. Complementary and Alternative Medicine in the United States. Washington, DC: The National Academies Press. doi: 10.17226/11182.
×

FIGURE 5-3 A model for CAM research and translation.

Suggested Citation:"5 State of Emerging Evidence on CAM." Institute of Medicine. 2005. Complementary and Alternative Medicine in the United States. Washington, DC: The National Academies Press. doi: 10.17226/11182.
×

the public’s health. For example, data from the Multiple Risk Factor Intervention Trial and the Framingham Heart Study revealed the importance of preventing “development of unfavorable levels of blood cholesterol and blood pressure, cigarette smoking, diabetes, and unfavorable body weight” as crucial to preventing clinical coronary heart disease (Greenland et al., 2003).

The ideal situation would be the inclusion of CAM-related questions in such studies as the National Health and Nutrition Examination Survey (NHANES) study, the Behavioral Risk Factor Surveillance Survey, the Framingham Heart Study, or the Nurses’ Health Study. This would enable investigators to examine the impact of CAM use over time on the health of individuals as well as the members of a particular cohort.

Finally, surveys that are explicitly intended to capture the real and the perceived adverse events associated with CAM use should be established. The results of such surveys could be used to identify high-priority research topics.

The information produced by the types of surveys described above is crucial to understanding the use of CAM therapies in the United States and prioritization of the CAM-related research portfolio. These surveys form a key component of the model illustrated in Figure 5-3. Although they are extremely valuable, large national surveys are unable to monitor emergent patterns of CAM use, the specifics of CAM use, and particular combinations of CAM use for particular purposes. Other types of data need to be collected to explore such things as how CAM use is associated with behavioral change or is related to risk and health-protective or health-promotive behaviors. Another key component of that model, the sentinel surveillance system, plays an important role in answering these types of questions by providing timely information about treatment trends that demand research and is described below.

Sentinel Surveillance System

Surveillance is defined as “ongoing systematic collection, analysis and interpretation of data and the distribution to those who need to know” (Thacker and Berkelman, 1988). Surveillance has many uses when it is well designed and implemented and can be used to

  • obtain quantitative estimates of the magnitude of a health problem;

  • portray the natural history of disease;

  • detect epidemics;

  • document the distribution and spread of a health event;

  • facilitate epidemiological and laboratory research;

Suggested Citation:"5 State of Emerging Evidence on CAM." Institute of Medicine. 2005. Complementary and Alternative Medicine in the United States. Washington, DC: The National Academies Press. doi: 10.17226/11182.
×
  • test a hypothesis;

  • evaluate control and prevention measures;

  • monitor changes in the prevalence of infectious agents;

  • monitor isolation activities;

  • monitor behavioral change-associated risk or prevention of illness;

  • monitor public receptiveness to marketing and public health messages—i.e., information;

  • detect changes in health care practices; and

  • plan (Thacker, 1992).

Groseclose and colleagues (2000) classify surveillance systems as either passive or active. Passive surveillance reporting systems can provide incomplete information because they depend on the voluntary reporting of data. Active surveillance systems, on the other hand, solicit data from selected sites for specific purposes, for example, the use of particular antibiotics for a condition like otitis media. A sentinel surveillance system can be either active or passive and is composed of selected sites that report information that may be generalizable to the population as a whole (Birkhead and Maylahn, 2000). Sentinel sites (both CAM and conventional medicine) might include practitioners, hospitals, specialty clinics, clinics that serve specific population subgroups, health maintenance organizations, and teaching hospitals that have agreed to report information. Member sites might serve either a common population defined by a particular set of health problems or a particular population (by age, ethnicity, location, etc.).

The value of sentinel surveillance systems can be seen in the public health system, in which they “have been particularly helpful in monitoring specific infections or designated classes of infections” (IOM, 2003a). The Emerging Infections Program, a surveillance system collaboration among the Centers for Disease Control and Prevention, state public health departments, and other public health partners, conducts surveillance for unexplained deaths and severe illnesses to “identify diseases and infectious agents, known and unknown, that can lead to severe illness or death” (IOM, 2003a).

For CAM, a sentinel surveillance system is important to monitoring changes in the American public’s use of CAM alone or in combination with conventional medicine, thereby highlighting high-priority areas for practice-based research.

The third major component of the model described in Figure 5-3 is practice-based research networks. The following discussion describes the value and importance of such networks both to effectiveness research and to the translation of validated therapies into practice.

Suggested Citation:"5 State of Emerging Evidence on CAM." Institute of Medicine. 2005. Complementary and Alternative Medicine in the United States. Washington, DC: The National Academies Press. doi: 10.17226/11182.
×

Practice-Based Research Networks

Nutting (1996) writes, “In the primary care setting, patients present with multiple problems: some are diseases, others are illnesses that may become diseases, and many are neither, yet all are important and can measurably decrease function and quality of life.” Although, Nutting was referring to conventional primary care, the description also holds for patients who seek CAM therapies or both CAM and conventional. He also describes the need to determine practice-relevant research questions, to develop research that draws on the strengths and experiences of practitioners, and to ensure that rigorous and multiple research methods are used for study, concluding that PBRNs are best able to incorporate these and other necessary elements. Nutting is not alone in his belief that PBRNs are vital to effective primary care. Other individuals and organizations also support the development of PBRNs to increase the research and evidence base in primary care (Fenton et al., 2001; Genel and Dobs, 2003; Green and Dovey, 2001; IOM, 1996; Lindbloom et al., 2004).

PBRNs are defined as “a group of ambulatory practices devoted principally to the primary care of patients, affiliated with each other (and often with an academic or professional organization) in order to investigate questions related to community-based practice” (AHRQ, 2001). The 1996 IOM report Primary Care: America’s Health in a New Era, states, “The committee sees practice-based research networks as a significant underpinning for studies in primary care, noting not only their attractiveness conceptually but the growing recognition of their value” (IOM, 1996). That report recommend that “the Department of Health and Human Services provide adequate and stable financial support to practice-based primary care research networks.”

According to Thomas et al. (2001), PBRNs emerged in the United Kingdom in the 1960s. Several regional networks were begun in the United States during the 1970s with the first national network established in 1981 (Lindbloom et al., 2004). The Federation of Practice Based Networks, established in 1997, advocates and builds capacity for practice-based research and works to facilitate communication and collaboration among networks (FPBRN, 2004).

Thomas and colleagues (2001) suggest that PBRNs are virtual organizations that to succeed need to specify membership criteria, accountability, and authority; address issues of governance; and evaluate activities. They have identified three network leadership types. In the first type, “practitioners develop their own ideas and the network is led by a peer group,” thus facilitating participation because the direction is provided by the practitioners involved. The second type uses a top-down approach with “strong institutional links and research projects led by experts.” This approach

Suggested Citation:"5 State of Emerging Evidence on CAM." Institute of Medicine. 2005. Complementary and Alternative Medicine in the United States. Washington, DC: The National Academies Press. doi: 10.17226/11182.
×

fosters rapid research of high quality. The third type of network is referred to as “whole systems,” in which the leadership is multidisciplinary. Network participants form coalitions including both research experts and novices. According to Thomas et al. (2001), the whole systems approach is good for producing cultural change because different enthusiasts in different parts of the health care system become involved.

Griffiths and colleagues (2000) describe different network organizational styles in the following manner:

… some have a hierarchical organisation with a strong centre, often at a university, leading satellite units or network members; others are less hierarchical with coordination and cooperation between satellite units and members as well as with the centre.

Despite differences in design and organization, Nutting et al. (1999) describe four central characteristics of all PBRNs:

  1. PBRNs capture health care events that reflect the selection and observer bias that characterize primary care in community-based patient populations.

  2. PBRNs provide access to the practice experience and care provided by full-time primary care clinicians.

  3. PBRNs focus their activities on practice-relevant research questions, apply appropriate multimethod research designs, and generally avoid the tendency to permit research methods to define the question.

  4. Networks strive for the systematic involvement of network clinicians in defining the research questions, participating in the study design, and interpreting study results.

PBRNs conduct studies that use both qualitative and quantitative methods ranging from observational studies to RCTs. For example, using observational data from The Direct Observation of Primary Care Study, Stange et al. (1998) found “that family physicians target preventive services toward patients most in need of them and use illness visits as opportunities for prevention”; a randomized control trial by Fleming et al. “provided the first direct evidence that brief physician intervention was effective in reducing alcohol use and utilization of health care services” (Nutting et al., 1999). Some networks focus on providing epidemiological data; others are concerned with the process of care (Griffiths et al., 2000).

A PBRN can provide information on the content and the practice patterns offered in various types of clinical settings, offer flexibility in collecting and analyzing data from a variety of perspectives (e.g., the practitioner and the patient), provide the opportunity to ask and answer practice-relevant questions, and study CAM treatments in the manner in which they

Suggested Citation:"5 State of Emerging Evidence on CAM." Institute of Medicine. 2005. Complementary and Alternative Medicine in the United States. Washington, DC: The National Academies Press. doi: 10.17226/11182.
×

are practiced. Once a PBRN is in place, the additional costs of mounting specific studies can be fairly low, and the level of preparedness engendered by its structure can allow the timely generation of research findings (Zarin et al., 1997). An additional benefit of PBRNs, especially in the area of CAM, which has historically lacked a research infrastructure, is their potential to provide places of learning, provide training in research, and, through direct involvement of practitioners with science, promote a climate of inquiry that both questions and increases the evidence underlying a particular practice.

Furthermore, according to Genel and Dobs (2003), PBRNs can facilitate the translation of research findings into practice. They assert that PBRNs address two of the greatest difficulties in translating findings into practice: the lack of communication between academic and practicing physicians and the failure to address practitioner needs in research. Genel and Dobs assert that practitioners must be trained in clinical research and that, as their familiarity with that research grows, they will be enthusiastic about the research effort and will be more likely to implement the research results in their own practices.

CAM Research Centers

The fourth major component of the research model proposed here is a CAM Research Center. Currently, NCCAM funds specialized research centers, each of which focuses on research in one particular area (see Appendix F for a list of these centers). For example, the center at the Oregon Health Sciences University focuses on CAM research in neurological disorders; the center at Columbia University investigates CAM use in aging; the center at the University of California at Los Angeles fosters research evaluating safety and efficacy of botanical dietary supplements; and the center at the University of Illinois at Chicago is studying botanical dietary supplements for women’s health.

The CAM research centers envisioned in the committee’s research framework differ from these specialized research centers in important ways. First, they are not restricted to one central focus. Rather they would facilitate the activities of the research networks across many topic areas. The centers in Figure 5-3 would propose (or seek input from the network for) specific research questions and protocols. They would help coordinate the design and implementation of these investigations; coordinate refinement of protocols and priorities over time; and supervise the analyses of data generated from these studies. Unlike NCCAM centers as currently structured and funded, these new centers require a much broader spectrum of expertise from both the conventional and CAM research and clinical communities because they would not be focused on one modality or one clinical

Suggested Citation:"5 State of Emerging Evidence on CAM." Institute of Medicine. 2005. Complementary and Alternative Medicine in the United States. Washington, DC: The National Academies Press. doi: 10.17226/11182.
×

problem. These centers must have the capacity to work with the networks to identify important questions and to design studies that are hypothesis generating and hypothesis testing.

The proposed centers coordinate data collection and analysis—bringing in information gleaned through longitudinal and periodic studies as well as data collected by the surveillance sites—and provide research and other training to network and surveillance participants as needed. In some instances, it is likely that they will serve in the more traditional role of a coordinating site for multisite observational or controlled trials, but this is not the main or sole purpose of their creation.

The committee believes strongly that the center team should be transdisciplinary, at a minimum composed of methodologists, social and behavioral scientists, and experienced integrative medicine practitioners. As the network evolves it is anticipated that individuals from other disciplines would join the coordinating team. This team should be committed to a participatory research process and the cannons of conducting good science.

These four major components, national surveys, a sentinel surveillance system, practice-based research networks, and CAM research centers, form the core of the research, reporting, and translation model developed by the committee and illustrated in Figure 5-3.

A Model for CAM Research and Surveillance

In the model for CAM research and surveillance shown in Figure 5-3 the left-hand hub represents the sentinel surveillance function. Sentinel surveillance sites would be responsible for the systematic collection and reporting of data on common and emerging patterns of CAM use as well as the use of both CAM and conventional medicine. Such information could be used to identify treatment trends that demand research. A sentinel surveillance approach to the collection of data on the use of CAM would complement the periodic population-based survey approach because data would be collected in an ongoing fashion in contexts subject to real-world contingencies and for a variety of different populations. The sentinel surveillance systems formed in different parts of the country would also provide data that allow the monitoring and analysis of regional and national trends in CAM use.

Another advantage of sentinel surveillance systems relates to the translation of research results into practice. That is, such systems allow the real-time monitoring of the impact of the information about various CAM treatments disseminated to the public and practitioners by asking such questions as How does information influence practice? and How does treatment affect behaviorial change? The information collected by the sentinel sites could then be reported to the CAM research center (center of

Suggested Citation:"5 State of Emerging Evidence on CAM." Institute of Medicine. 2005. Complementary and Alternative Medicine in the United States. Washington, DC: The National Academies Press. doi: 10.17226/11182.
×

Figure 5-3), where it would be reviewed to determine whether emerging patterns that would be useful to study may exist.

The right-hand hub of Figure 5-3 is the PBRN. Network participants would be recruited from many disciplines and would include both CAM and conventional medicine practitioners. It is anticipated that this network will conduct practice-based participatory research. Practitioners involved in the PBRN will learn research study designs through their preparation for and participation in network search activities. Additionally, these practitioners will be in a position to guide the development of new instruments and outcome measures of relevance to the community.

In the model illustrated in Figure 5-3, the CAM research center would work with the PBRN to develop a consensus on how best to test the effectiveness and safety of treatments and bundles of behavior associated with treatment approaches. Together, the center and the network would

  • Select target conditions to be evaluated

  • Develop a protocol designed to capture

    • Health care practices engaged in by patients as self-care

    • Health care administered or prescribed by practitioners

    • Interactions between self-care and practitioner care and between conventional medical care and CAM care

    • Notable lifestyle changes that may effect health status, e.g., interactions between medication use and special diets the population has adopted

    • Exposure of the population to marketing and public health messages

  • Identify sites attending to patients (by both CAM and conventional medicine) with the targeted conditions

  • Organize the data collection on the basis of an agreed-upon sampling procedure

  • Develop an effectiveness study or intervention activity linked to the surveillance data

It is the committee’s belief that the model for research and translation illustrated and described above would provide a coordinated mechanism directed at answering the myriad questions about CAM use, such as Who is using CAM and why are they doing so? and Are CAM therapies safe, effective, and cost-effective? The committee strongly urges NIH and other public agencies to provide the support necessary to develop and implement such a model.

Suggested Citation:"5 State of Emerging Evidence on CAM." Institute of Medicine. 2005. Complementary and Alternative Medicine in the United States. Washington, DC: The National Academies Press. doi: 10.17226/11182.
×

CONCLUSIONS AND RECOMMENDATIONS

Both Chapter 2 and this chapter have presented information on what is known about CAM and where the gaps in knowledge exist. Chapter 2 presented the committee’s recommendations on ways to address the gaps in knowledge discussed in that chapter. These gaps in knowledge are reviewed below:

  • The information available about the motivations for CAM use indicates that pursuit of wellness is a major impetus; however, the extent to which CAM use is a trigger for positive behavioral change is unknown and constitutes an important research issue.

  • Existing surveys provide little information about how the use of CAM therapies is initiated; that is, are they self-initiated, provider initiated, provider administered, or self-administered?

  • There are virtually no data on adherence to CAM treatment or self-treatment with CAM. This information is crucial to assessments of the real-world effectiveness and safety of CAM therapies and their use.

  • Longitudinal studies are needed to clarify people’s trajectories of CAM use and those factors that influence upward and downward slopes in use.

  • There is little research on how the public obtains information about CAM therapies: what types of information are deemed credible, marginal, and spurious; how does the public understand the information in terms of risks and benefits; how do such perceptions inform decision making; and what do members of the public expect their providers to tell them?

This chapter has discussed the emerging evidence about CAM therapies, including sources of information (MEDLINE, The Cochrane Library); summarized the systematic reviews that have been conducted; examined the need for high-quality studies; and explored the gaps in evidence. The gaps discussed include the paucity of clinical research in which CAM interventions are compared with each other and with conventional medicine therapies, the need for expanded basic research to include areas other than botanicals, the lack of cost-effectiveness research, the need for cross-disciplinary and transdisciplinary research, and the importance of drawing established scientists to the field of CAM research. The committee has proposed a research framework to address these gaps as well as conduct the kinds of research recommended in Chapter 2. This framework includes a sentinel surveillance system, PBRNs, and CAM research centers that can incorporate information from national surveys (both periodic and longitudinal) and facilitate the work of the PBRNs.

Suggested Citation:"5 State of Emerging Evidence on CAM." Institute of Medicine. 2005. Complementary and Alternative Medicine in the United States. Washington, DC: The National Academies Press. doi: 10.17226/11182.
×

The committee believes that a research model such as the one described in this chapter, if it is adequately funded and implemented, will help provide additional understanding of the vast and varied field of CAM. As part of the following recommendation, the committee specifies the need for oversampling of racial and ethnic minorities in surveys. Many racial and ethnic minorities constitute a comparatively small proportion of the total U.S. population or are concentrated in a small number of geographic areas. In both situations, the number of individuals in a specific racial or ethnic group who will be selected in a nationwide random sample is typically too small to provide the basis for statistically reliable estimates for that population. Therefore, the only way to obtain meaningful results for such minority groups and to allow comparisons with the majority non-Hispanic white population is to oversample these groups, that is, to select the sample in a fashion that ensures that the proportion of individuals from these minority groups in the sample is larger than their proportion in the overall U.S. population.

The committee recommends that the National Institutes of Health and other public agencies provide the support necessary to

  • develop and implement a sentinel surveillance system, practice-based research networks, and CAM research centers to facilitate the work of the networks;

  • include CAM-relevant questions in federally funded health care surveys (e.g., the National Health Interview Survey) and in ongoing longitudinal cohort studies (e.g., the Nurses’ Health Study and the Framingham Heart Study); and

  • implement periodic comprehensive, representative, national surveys to assess the changes in the prevalence, patterns, perceptions, and costs of therapy use (both CAM and conventional medicine), with oversampling of ethnic minorities.

REFERENCES

AHRQ (Agency for Healthcare Research and Quality). 2001. Primary Care Practice-Based Research Networks. [Online]. Available: http://www.ahrq.gov/research/pbrnfact.htm [accessed June 16, 2004].

Allen C, Clark M. 2003. International Activity Within Collaborative Review Groups. In: 11th Cochrane Colloquium: Evidence, Healthcare and Culture. Oxford, UK: U.K. Cochrane Center. P. 44.

Atallah AN, Hofmeyr GJ, Duley L. Calcium supplementation during pregnancy for preventing hypertensive disorders and related problems. Cochrane Database Syst Rev 2004; (1).


Barnes PM, Powell-Griner E, McFann K, Nahin RL. 2004. Complementary and alternative medicine use among adults: United States, 2002. Vital Health Stat 343:1–19 (advance data).

Suggested Citation:"5 State of Emerging Evidence on CAM." Institute of Medicine. 2005. Complementary and Alternative Medicine in the United States. Washington, DC: The National Academies Press. doi: 10.17226/11182.
×

Bausell RB, Lee WL, Li YF, Soeken KS, Berman BM. 2004. Larger effect sizes were associated with higher quality ratings in complementary and alternative medicine randomized controlled trials. J Clin Epidemiol 57(5):438–446.

Beckles Willson N, Elliott TM, Everard ML. Omega-3 fatty acids (from fish oils) for cystic fibrosis. Cochrane Database Syst Rev 2004; (1).

Birkhead GS, Maylahn CM. 2000. State and Local Public Health Surveillance in Principles and Practice of Public Health Surveillance. Teutsch SM, Churchill RE, eds. New York: Oxford University Press. Pp. 253–287.

Brosseau L, Yonge KA, Robinson V, Marchand S, Judd M, Wells G, Tugwell P. Transcutaneous electrical nerve stimulation (TENS) for the treatment of rheumatoid arthritis in the hand. Cochrane Database Syst Rev 2004; (1).

Busby MG, Jeffcoat AR, Bloedon LT, Koch MA, Black T, Dix KJ, Heizer WD, Thomas BF, Hill JM, Crowell JA, Zeisel SH. 2002. Clinical characteristics and pharmacokinetics of purified soy isoflavones: Single-dose administration to healthy men. Am J Clin Nutr 75(1):126–136.


Caraballoso M, Sacristan M, Serra C, Bonfill X. 2003. Drugs for preventing lung cancer in healthy people. Cochrane Database Syst Rev 2:CD002141.

Caraballoso M, Sacristan M, Serra C, Bonfill X. Drugs for preventing lung cancer in healthy people. Cochrane Database Syst Rev 2004; (1).


D’Souza RM, D’Souza R. Vitamin A for treating measles in children. Cochrane Database Syst Rev 2004; (1).

Darlow B, Austin N. Selenium supplementation to prevent short-term morbidity in preterm neonates. Cochrane Database Syst Rev 2004; (4).

Darlow BA, Graham PJ. Vitamin A supplementation for preventing morbidity and mortality in very low birthweight infants. Cochrane Database Syst Rev 2004; (4).

Davidson RJ, Kabat-Zinn J, Schumacher J, Rosenkranz M, Muller D, Santorelli SF, Urbanowski F, Harrington A, Bonus K, Sheridan JF. 2003. Alterations in brain and immune function produced by mindfulness meditation . Psychosom Med 65(4):564–570.

Dickersin K, Manheimer E. 1998. The Cochrane Collaboration: Evaluation of health care and services using systematic reviews of the results of randomized controlled trials. Clin Obstet Gynecol 41(2):315–331.

Dickersin K, Scherer R, Lefebvre C. 1994. Identifying relevant studies for systematic reviews. BMJ 309(6964):1286–1291.

Douglas RM, Chalker EB, Treacy B. Vitamin C for preventing and treating the common cold. Cochrane Database Syst Rev 2004; (1).


Egger M, Juni P, Bartlett C, Holenstein F, Sterne J. 2003. How important are comprehensive literature searches and the assessment of trial quality in systematic reviews? Empirical study. Health Technol Assess 7(1):1–76.

Eisenberg DM, Davis RB, Ettner SL, Appel S, Wilkey S, Van Rompay M, Kessler RC. 1998. Trends in alternative medicine use in the United States, 1990-1997: Results of a follow-up national survey. JAMA 280(18):1569–1575.

Evans JR. Antioxidant vitamin and mineral supplements for age-related macular degeneration. Cochrane Database Syst Rev 2004; (1).

Ezzo J, Bausell B, Moerman DE, Berman B, Hadhazy V. 2001. Reviewing the reviews. How strong is the evidence? How clear are the conclusions? Int J Technol Assess Health Care 17(4):457–466.


Farmer A, Montori V, Dinneen S, Clar C. Fish oil in people with type 2 diabetes mellitus. Cochrane Database Syst Rev 2004; (1).

Fenton E, Harvey J, Griffiths F, Wild A, Sturt J. 2001. Reflections from organization science on the development of primary health care research networks. Fam Pract 18(5):540–544.

Suggested Citation:"5 State of Emerging Evidence on CAM." Institute of Medicine. 2005. Complementary and Alternative Medicine in the United States. Washington, DC: The National Academies Press. doi: 10.17226/11182.
×

FPBRN (Federation of Practice Based Research Networks). 2004. Federation of Practice Based Research Networks Homepage. [Online]. Available: http://www.aafp.org/x19544.xml [accessed April 14, 2004].

Furlan AD, Brosseau L, Imamura M, Irvin E. Massage for low back pain. Cochrane Database Syst Rev 2004; (1).

Friedman R, Sobel D, Myers P, Caudill M, Benson H. 1995. Behavioral medicine, clinical health psychology, and cost offset. Health Psychol 14(6):509–518.


Genel M, Dobs A. 2003. Translating clinical research into practice: Practice-based research networks—a promising solution. J Investig Med 51(2):64–71.

Green LA, Dovey SM. 2001. Practice based primary care research networks. They work and are ready for full development and support. BMJ 322(7286):567–568.

Greenland P, Knoll MD, Stamler J, Neaton JD, Dyer AR, Garside EB, Wilson PW. 2003. Major risk factors as antecedents of fatal and nonfatal coronary heart disease events. JAMA 290:891–897.

Griffiths F, Wild A, Harvey J, Fenton E. 2000. The productivity of primary care research networks. Br J Gen Pract 50(460):913–915.

Grol R, Grimshaw J. 2003. From best evidence to best practice: Effective implementation of change in patients’ care. Lancet 362(9391):1225–1230.

Groseclose SL, Sullivan KM, Gibbs NP, Knowles CM. 2000. Management of the Surveillance Information System and Quality Control of Data. In: Teutsch SM, Churchill RE, eds. Principles and Practice of Public Health Surveillance. New York: Oxford University Press. Pp. 95–112.


Health Forum/American Hospital Association. 2003. 2003 Complementary and Alternative Medicine Survey. [Online]. Available: http://www.hospitalconnect.com/aha/resource_center/statistics/Complementary%20and%20Alternative%20Medicine%20Survey.html [accessed June 16, 2004].

Herxheimer A, Petrie KJ. Melatonin for preventing and treating jet lag. Cochrane Database Syst Rev 2004; (1).

Homik J, Suarez-Almazor ME, Shea B, Cranney A, Wells G, Tugwell P. Calcium and vitamin D for corticosteroid-induced osteoporosis. Cochrane Database Syst Rev 2004; (1).

Howlett A, Ohlsson A. Inositol for respiratory distress syndrome in preterm infants. Cochrane Database Syst Rev 2004; (1).

Hulme J, Robinson V, DeBie R, Wells G, Judd M, Tugwell P. Electromagnetic fields for the treatment of osteoarthritis. Cochrane Database Syst Rev 2004; (1).

Hypericum Depression Trial Study Group. 2002. Effect of Hypericum perforatum (St John’s wort) in major depressive disorder: A randomized controlled trial. JAMA 287(14):1807–1814.


IOM (Institute of Medicine). 1996. Primary Care: America’s Health in a New Era. Washington, DC: National Academy Press.

IOM. 2003a. The Future of the Public’s Health. Washington, DC: The National Academies Press.

IOM. 2003b. Who Will Keep the Public Healthy? Washington, DC: The National Academies Press.


Jadad AR, Moore RA, Carroll D, Jenkinson C, Reynolds DJ, Gavaghan DJ, McQuay HJ. 1996. Assessing the quality of reports of randomized clinical trials: Is blinding necessary? Control Clin Trials 17(1):1–12.

Jadad AR, Cook DJ, Jones A, Klassen TP, Tugwell P, Moher M, Moher D. 1998. Methodology and reports of systematic reviews and meta-analyses: A comparison of Cochrane reviews with articles published in paper-based journals. JAMA 280(3):278–280.

Suggested Citation:"5 State of Emerging Evidence on CAM." Institute of Medicine. 2005. Complementary and Alternative Medicine in the United States. Washington, DC: The National Academies Press. doi: 10.17226/11182.
×

Jadad AR, Moher M, Browman GP, Booker L, Sigouin C, Fuentes M, Stevens R. 2000. Systematic reviews and meta-analyses on treatment of asthma: Critical evaluation. BMJ 320(7234):537–540.

Jepson RG, Mihaljevic L, Craig J. Cranberries for preventing urinary tract infections. Cochrane Database Syst Rev 2004; (1).

Juni P, Altman DG, Egger M. 2001. Systematic reviews in health care: Assessing the quality of controlled clinical trials. BMJ 323(7303):42–46.


Kennelly EJ, Baggett S, Nuntanakorn P, Ososki AL, Mori SA, Duke J, Coleton M, Kronenberg F. 2002. Analysis of thirteen populations of black cohosh for formononetin. Phytomedicine 9(5):461–467.


Lee WL, Bausell RB, Berman BM. 2001. The growth of health-related meta-analyses published from 1980 to 2000. Evaluation and the Health Professions 24(3):327–335.

Lindbloom EJ, Ewigman BG, Hickner JM. 2004. Practice-based research networks: The laboratories of primary care research. Med Care 42(4 Suppl):III45–III49.

Linde K, Hondras M, Vickers A, ter Riet G, Melchart D. 2001a. Systematic reviews of complementary therapies—An annotated bibliography. Part 3: Homeopathy. BMC Complemen Altern Med 1(1):4 (electronic publication).

Linde K, Jonas WB, Melchart D, Willich S. 2001b. The methodological quality of randomized controlled trials of homeopathy, herbal medicines and acupuncture. Int J Epidemiol 30(3):526–531.

Linde K, ter Riet G, Hondras M, Vickers A, Saller R, Melchart D. 2001c. Systematic reviews of complementary therapies—An annotated bibliography. Part 2: Herbal medicine. BMC Complement Altern Med 1(1):5 (electronic publication).

Linde K, Vickers A, Hondras M, ter Riet G, Thormahlen J, Berman B, Yelchart D. 2001d. Systematic reviews of complementary therapies—An annotated bibliography. Part 1: Acupuncture. BMC Complement Altern Med 2001;1(1):3 (electronic publication).

Linde K, Mulrow CD. St John’s wort for depression. Cochrane Database Syst Rev 2004; (1).

Little CV, Parsons T. Herbal therapy for treating osteoarthritis. Cochrane Database Syst Rev 2004; (1).

Lorig KR, Sobel DS, Stewart AL, Brown BW Jr, Bandura A, Ritter P, Gonzalez VM, Laurent DD, Holman HR. 1999. Evidence suggesting that a chronic disease self-management program can improve health status while reducing hospitalization: A randomized trial. Med Care 37(1):5–14.

Lumley J, Watson L, Watson M, Bower C. Periconceptional supplementation with folate and/ or multivitamins for preventing neural tube defects. Cochrane Database Syst Rev 2004; (1).


MacPherson H, White A, Cummings M, Jobst K, Rose K, Niemtzow R. 2001. Standards for reporting interventions in controlled trials of acupuncture: The STRICTA recommendations. Complement Ther Med 9(4):246–249.

Mahomed K. Folate supplementation in pregnancy. Cochrane Database Syst Rev 2004; (1).

Mahomed K. Iron and folate supplementation in pregnancy. Cochrane Database Syst Rev 2004; (1).

Mahomed K. Iron supplementation in pregnancy. Cochrane Database Syst Rev 2004; (1).

Manheimer E, Anderson D. 2002. Survey of public information about ongoing clinical trials funded by industry: Evaluation of completeness and accessibility. BMJ 325(7363):528–531.

McAlister FA, Laupacis A, Wells GA, Sackett DL. 1999. Users’ guides to the medical literature: XIX. Applying clinical trial results B. Guidelines for determining whether a drug is exerting (more than) a class effect. JAMA 282(14):1371–1377.

Melchart D, Linde K, Fischer P, Berman B, White A, Vickers A, Allais G. Acupuncture for idiopathic headache. Cochrane Database Syst Rev 2004; (1).

Suggested Citation:"5 State of Emerging Evidence on CAM." Institute of Medicine. 2005. Complementary and Alternative Medicine in the United States. Washington, DC: The National Academies Press. doi: 10.17226/11182.
×

Melchart D, Linde K, Fischer P, Kaesmayr J. Echinacea for preventing and treating the common cold. Cochrane Database Syst Rev 2004; (1).

Moher D. 2002. The CONSORT guidelines: Improving the quality of research. Consolidated Standards of Reporting Trials. Interview by Bonnie Horrigan. Altern Ther Health Med 8(3):103–108.

Moher D, Soeken K, Sampson M, Ben-Porat L, Berman B. 2002. Assessing the quality of reports of systematic reviews in pediatric complementary and alternative medicine. BMC Pediatr 2(1):3.


National Institutes of Health. 1998. Acupuncture. NIH Consensus Conference. JAMA 280(17):1518–1524.

National Library of Medicine. 2002. MEDLINE Fact Sheet. [Online]. Available: http://www.nlm.nih.gov/pubs/factsheets/medline.html [accessed June 16, 2004].

NCCAM (National Center for Complementary and Alternative Medicine). 2004. Centers for Research on Complementary and Alternative Medicine (CRC) Program FY 2004 Research Priorities. [Online]. Available: http://nccam.nih.gov/research/priorities/index.htm [accessed June 16, 2004].

Nutting PA. 1996. Practice-based research networks: Building the infrastructure of primary care research. J Fam Pract 42(2):199–203.

Nutting PA, Beasley JW, Werner JJ. 1999. Practice-based research networks answer primary care questions. JAMA 281(8):686–688.


Ortiz Z, Shea B, Suarez Almazor M, Moher D, Wells G, Tugwell P. Folic acid and folinic acid for reducing side effects in patients receiving methotrexate for rheumatoid arthritis. Cochrane Database Syst Rev 2004; (1).

Osiri M, Welch V, Brosseau L, Shea B, McGowan J, Tugwell P, Wells G. Transcutaneous electrical nerve stimulation for knee osteoarthritis. Cochrane Database Syst Rev 2004; (1).


Pittler MH, Ernst E. Horse chestnut seed extract for chronic venous insufficiency. Cochrane Database Syst Rev 2004; (1).

Pittler MH, Ernst E. Kava extract for treating anxiety. Cochrane Database Syst Rev 2004; (1).

Pollo A, Amanzio M, Arslanian A, Casadio C, Maggi G, Benedetti F. 2001. Response expectancies in placebo analgesia and their clinical relevance. Pain 93(1):77–84.

Proctor ML, Hing W, Johnson TC, Murphy PA. Spinal manipulation for primary and secondary dysmenorrhoea. Cochrane Database Syst Rev 2004; (1).

Proctor ML, Smith CA, Farquhar CM, Stones RW. Transcutaneous electrical nerve stimulation and acupuncture for primary dysmenorrhoea. Cochrane Database Syst Rev 2004; (1).


Schulz KF, Chalmers I, Hayes RJ, Altman DG. 1995. Empirical evidence of bias. Dimensions of methodological quality associated with estimates of treatment effects in controlled trials. JAMA 273(5):408–412.

Shea B, Wells G, Granney A, Zytaruk N, Robinson V, Griffen L, Hamel C, Ortiz Z, Peterson J, Adachi J, Tugwell P, Guyatt G. Calcium supplementation on bone loss in post-menopausal women. Cochrane Database Syst Rev 2004; (1).

Shekelle P, Eccles MP, Grimshaw JM, Woolf SH. 2001. When should clinical guidelines be updated? BMJ 323(7305):155–157.

Shiffman RN, Shekelle P, Overhage JM, Slutsky J, Grimshaw J, Deshpande AM. 2003. Standardized reporting of clinical practice guidelines: A proposal from the Conference on Guideline Standardization. Ann Intern Med 139(6):493–498.

Silagy CA, Stead LF, Lancaster T. 2001. Use of systematic reviews in clinical practice guidelines: Case study of smoking cessation. BMJ 323(7317):833–836.

Stange KC, Zyzanski SJ, Jaen CR, Callahan EJ, Kelly RB, Gillanders WR, Shank JC, Chao J, Medalie JH, Miller WL, Crabtree BF, Flocke SA, Gilchrest VJ, Langa DM, Goodwin MA. 1998. Illuminating the “black box”. A description of 4454 patient visits to 138 family physicians. J Fam Pract 46:377–389.

Suggested Citation:"5 State of Emerging Evidence on CAM." Institute of Medicine. 2005. Complementary and Alternative Medicine in the United States. Washington, DC: The National Academies Press. doi: 10.17226/11182.
×

Suresh GK, Davis JM, Soll RF. Superoxide dismutase for preventing chronic lung disease in mechanically ventilated preterm infants. Cochrane Database Syst Rev 2004; (1).


Tabet N, Birks J, Grimley Evans J. Vitamin E for Alzheimer’s disease. Cochrane Database Syst Rev 2004; (1).

Tang JL, Wong TW. 1998. The need to evaluate the clinical effectiveness of traditional Chinese medicine. Hong Kong Med J 4(2):208–210.

Taylor JA, Weber W, Standish L, Quinn H, Goesling J, McGann M, Calabrese C. 2003. Efficacy and safety of echinacea in treating upper respiratory tract infections in children: A randomized controlled trial. JAMA 290(21):2824–2830.

Taylor MJ, Carney S, Geddes J, Goodwin G. Folate for depressive disorders. Cochrane Database Syst Rev 2004; (1).

ter Riet G, de Craen AJ, de Boer A, Kessels AG. 1998. Is placebo analgesia mediated by endogenous opioids? A systematic review. Pain 76(3):273–275.

Thacker SB. 1992. Les principes et la pratique de la surveillance en santé publique; L’utilization des données en santé publique [The principles and practice of surveillance in public health; The use of data in public health]. Santé Publique 1992:43–44.

Thacker SB, Berkelman RL. 1988. Public health surveillance in the United States. Epidemiol Rev 10:164–190.

Thomas P, Griffiths F, Kai J, O’Dwyer A. 2001. Networks for research in primary health care. BMJ 322(7286):588–590.

Towheed TE, Anastassiades TP, Shea B, Houpt J, Welch V, Hochberg MC. Glucosamine therapy for treating osteoarthritis. Cochrane Database Syst Rev 2004; (1).


Vickers A. 1998. Bibliometric analysis of randomized trials in complementary medicine. Complement Ther Med 6:185–189.


White AR, Ernst E. 2000. Economic analysis of complementary medicine: A systematic review. Complement Ther Med 8(2):111–118.

Wilson ML, Murphy PA. Herbal and dietary therapies for primary and secondary dysmenorrhoea. Cochrane Database Syst Rev 2004; (1).

Wilt T, Ishani A, MacDonald R, Rutks I, Stark G. Pygeum africanum for benign prostatic hyperplasia. Cochrane Database Syst Rev 2004; (1).

Wilt T, Ishani A, MacDonald R, Stark G, Mulrow C, Lau J. Beta-sitosterols for benign prostatic hyperplasia. Cochrane Database Syst Rev 2004; (1).

Wilt T, Ishani A, Stark G, MacDonald R, Mulrow C, Lau J. Serenoa repens for benign prostatic hyperplasia. Cochrane Database Syst Rev 2004; (1).

Wilt T, MacDonald R, Ishani A, Rutks I, Stark G. Cernilton for benign prostatic hyperplasia. Cochrane Database Syst Rev 2004; (1).


Zarin D, Pincus HA, West J, McIntyre JS. 1997. Practice-based research in psychiatry. Am J Psychiatry 154(9):1199–1208.

Suggested Citation:"5 State of Emerging Evidence on CAM." Institute of Medicine. 2005. Complementary and Alternative Medicine in the United States. Washington, DC: The National Academies Press. doi: 10.17226/11182.
×
Page 129
Suggested Citation:"5 State of Emerging Evidence on CAM." Institute of Medicine. 2005. Complementary and Alternative Medicine in the United States. Washington, DC: The National Academies Press. doi: 10.17226/11182.
×
Page 130
Suggested Citation:"5 State of Emerging Evidence on CAM." Institute of Medicine. 2005. Complementary and Alternative Medicine in the United States. Washington, DC: The National Academies Press. doi: 10.17226/11182.
×
Page 131
Suggested Citation:"5 State of Emerging Evidence on CAM." Institute of Medicine. 2005. Complementary and Alternative Medicine in the United States. Washington, DC: The National Academies Press. doi: 10.17226/11182.
×
Page 132
Suggested Citation:"5 State of Emerging Evidence on CAM." Institute of Medicine. 2005. Complementary and Alternative Medicine in the United States. Washington, DC: The National Academies Press. doi: 10.17226/11182.
×
Page 133
Suggested Citation:"5 State of Emerging Evidence on CAM." Institute of Medicine. 2005. Complementary and Alternative Medicine in the United States. Washington, DC: The National Academies Press. doi: 10.17226/11182.
×
Page 134
Suggested Citation:"5 State of Emerging Evidence on CAM." Institute of Medicine. 2005. Complementary and Alternative Medicine in the United States. Washington, DC: The National Academies Press. doi: 10.17226/11182.
×
Page 135
Suggested Citation:"5 State of Emerging Evidence on CAM." Institute of Medicine. 2005. Complementary and Alternative Medicine in the United States. Washington, DC: The National Academies Press. doi: 10.17226/11182.
×
Page 136
Suggested Citation:"5 State of Emerging Evidence on CAM." Institute of Medicine. 2005. Complementary and Alternative Medicine in the United States. Washington, DC: The National Academies Press. doi: 10.17226/11182.
×
Page 137
Suggested Citation:"5 State of Emerging Evidence on CAM." Institute of Medicine. 2005. Complementary and Alternative Medicine in the United States. Washington, DC: The National Academies Press. doi: 10.17226/11182.
×
Page 138
Suggested Citation:"5 State of Emerging Evidence on CAM." Institute of Medicine. 2005. Complementary and Alternative Medicine in the United States. Washington, DC: The National Academies Press. doi: 10.17226/11182.
×
Page 139
Suggested Citation:"5 State of Emerging Evidence on CAM." Institute of Medicine. 2005. Complementary and Alternative Medicine in the United States. Washington, DC: The National Academies Press. doi: 10.17226/11182.
×
Page 140
Suggested Citation:"5 State of Emerging Evidence on CAM." Institute of Medicine. 2005. Complementary and Alternative Medicine in the United States. Washington, DC: The National Academies Press. doi: 10.17226/11182.
×
Page 141
Suggested Citation:"5 State of Emerging Evidence on CAM." Institute of Medicine. 2005. Complementary and Alternative Medicine in the United States. Washington, DC: The National Academies Press. doi: 10.17226/11182.
×
Page 142
Suggested Citation:"5 State of Emerging Evidence on CAM." Institute of Medicine. 2005. Complementary and Alternative Medicine in the United States. Washington, DC: The National Academies Press. doi: 10.17226/11182.
×
Page 143
Suggested Citation:"5 State of Emerging Evidence on CAM." Institute of Medicine. 2005. Complementary and Alternative Medicine in the United States. Washington, DC: The National Academies Press. doi: 10.17226/11182.
×
Page 144
Suggested Citation:"5 State of Emerging Evidence on CAM." Institute of Medicine. 2005. Complementary and Alternative Medicine in the United States. Washington, DC: The National Academies Press. doi: 10.17226/11182.
×
Page 145
Suggested Citation:"5 State of Emerging Evidence on CAM." Institute of Medicine. 2005. Complementary and Alternative Medicine in the United States. Washington, DC: The National Academies Press. doi: 10.17226/11182.
×
Page 146
Suggested Citation:"5 State of Emerging Evidence on CAM." Institute of Medicine. 2005. Complementary and Alternative Medicine in the United States. Washington, DC: The National Academies Press. doi: 10.17226/11182.
×
Page 147
Suggested Citation:"5 State of Emerging Evidence on CAM." Institute of Medicine. 2005. Complementary and Alternative Medicine in the United States. Washington, DC: The National Academies Press. doi: 10.17226/11182.
×
Page 148
Suggested Citation:"5 State of Emerging Evidence on CAM." Institute of Medicine. 2005. Complementary and Alternative Medicine in the United States. Washington, DC: The National Academies Press. doi: 10.17226/11182.
×
Page 149
Suggested Citation:"5 State of Emerging Evidence on CAM." Institute of Medicine. 2005. Complementary and Alternative Medicine in the United States. Washington, DC: The National Academies Press. doi: 10.17226/11182.
×
Page 150
Suggested Citation:"5 State of Emerging Evidence on CAM." Institute of Medicine. 2005. Complementary and Alternative Medicine in the United States. Washington, DC: The National Academies Press. doi: 10.17226/11182.
×
Page 151
Suggested Citation:"5 State of Emerging Evidence on CAM." Institute of Medicine. 2005. Complementary and Alternative Medicine in the United States. Washington, DC: The National Academies Press. doi: 10.17226/11182.
×
Page 152
Suggested Citation:"5 State of Emerging Evidence on CAM." Institute of Medicine. 2005. Complementary and Alternative Medicine in the United States. Washington, DC: The National Academies Press. doi: 10.17226/11182.
×
Page 153
Suggested Citation:"5 State of Emerging Evidence on CAM." Institute of Medicine. 2005. Complementary and Alternative Medicine in the United States. Washington, DC: The National Academies Press. doi: 10.17226/11182.
×
Page 154
Suggested Citation:"5 State of Emerging Evidence on CAM." Institute of Medicine. 2005. Complementary and Alternative Medicine in the United States. Washington, DC: The National Academies Press. doi: 10.17226/11182.
×
Page 155
Suggested Citation:"5 State of Emerging Evidence on CAM." Institute of Medicine. 2005. Complementary and Alternative Medicine in the United States. Washington, DC: The National Academies Press. doi: 10.17226/11182.
×
Page 156
Suggested Citation:"5 State of Emerging Evidence on CAM." Institute of Medicine. 2005. Complementary and Alternative Medicine in the United States. Washington, DC: The National Academies Press. doi: 10.17226/11182.
×
Page 157
Suggested Citation:"5 State of Emerging Evidence on CAM." Institute of Medicine. 2005. Complementary and Alternative Medicine in the United States. Washington, DC: The National Academies Press. doi: 10.17226/11182.
×
Page 158
Suggested Citation:"5 State of Emerging Evidence on CAM." Institute of Medicine. 2005. Complementary and Alternative Medicine in the United States. Washington, DC: The National Academies Press. doi: 10.17226/11182.
×
Page 159
Suggested Citation:"5 State of Emerging Evidence on CAM." Institute of Medicine. 2005. Complementary and Alternative Medicine in the United States. Washington, DC: The National Academies Press. doi: 10.17226/11182.
×
Page 160
Suggested Citation:"5 State of Emerging Evidence on CAM." Institute of Medicine. 2005. Complementary and Alternative Medicine in the United States. Washington, DC: The National Academies Press. doi: 10.17226/11182.
×
Page 161
Suggested Citation:"5 State of Emerging Evidence on CAM." Institute of Medicine. 2005. Complementary and Alternative Medicine in the United States. Washington, DC: The National Academies Press. doi: 10.17226/11182.
×
Page 162
Suggested Citation:"5 State of Emerging Evidence on CAM." Institute of Medicine. 2005. Complementary and Alternative Medicine in the United States. Washington, DC: The National Academies Press. doi: 10.17226/11182.
×
Page 163
Suggested Citation:"5 State of Emerging Evidence on CAM." Institute of Medicine. 2005. Complementary and Alternative Medicine in the United States. Washington, DC: The National Academies Press. doi: 10.17226/11182.
×
Page 164
Suggested Citation:"5 State of Emerging Evidence on CAM." Institute of Medicine. 2005. Complementary and Alternative Medicine in the United States. Washington, DC: The National Academies Press. doi: 10.17226/11182.
×
Page 165
Suggested Citation:"5 State of Emerging Evidence on CAM." Institute of Medicine. 2005. Complementary and Alternative Medicine in the United States. Washington, DC: The National Academies Press. doi: 10.17226/11182.
×
Page 166
Suggested Citation:"5 State of Emerging Evidence on CAM." Institute of Medicine. 2005. Complementary and Alternative Medicine in the United States. Washington, DC: The National Academies Press. doi: 10.17226/11182.
×
Page 167
Next: 6 An Ethical Framework for CAM Research, Practice, and Policy »
Complementary and Alternative Medicine in the United States Get This Book
×
Buy Hardback | $65.00 Buy Ebook | $49.99
MyNAP members save 10% online.
Login or Register to save!
Download Free PDF

Integration of complementary and alternative medicine therapies (CAM) with conventional medicine is occurring in hospitals and physicians offices, health maintenance organizations (HMOs) are covering CAM therapies, insurance coverage for CAM is increasing, and integrative medicine centers and clinics are being established, many with close ties to medical schools and teaching hospitals. In determining what care to provide, the goal should be comprehensive care that uses the best scientific evidence available regarding benefits and harm, encourages a focus on healing, recognizes the importance of compassion and caring, emphasizes the centrality of relationship-based care, encourages patients to share in decision making about therapeutic options, and promotes choices in care that can include complementary therapies where appropriate.

Numerous approaches to delivering integrative medicine have evolved. Complementary and Alternative Medicine in the United States identifies an urgent need for health systems research that focuses on identifying the elements of these models, the outcomes of care delivered in these models, and whether these models are cost-effective when compared to conventional practice settings.

It outlines areas of research in convention and CAM therapies, ways of integrating these therapies, development of curriculum that provides further education to health professionals, and an amendment of the Dietary Supplement Health and Education Act to improve quality, accurate labeling, research into use of supplements, incentives for privately funded research into their efficacy, and consumer protection against all potential hazards.

  1. ×

    Welcome to OpenBook!

    You're looking at OpenBook, NAP.edu's online reading room since 1999. Based on feedback from you, our users, we've made some improvements that make it easier than ever to read thousands of publications on our website.

    Do you want to take a quick tour of the OpenBook's features?

    No Thanks Take a Tour »
  2. ×

    Show this book's table of contents, where you can jump to any chapter by name.

    « Back Next »
  3. ×

    ...or use these buttons to go back to the previous chapter or skip to the next one.

    « Back Next »
  4. ×

    Jump up to the previous page or down to the next one. Also, you can type in a page number and press Enter to go directly to that page in the book.

    « Back Next »
  5. ×

    Switch between the Original Pages, where you can read the report as it appeared in print, and Text Pages for the web version, where you can highlight and search the text.

    « Back Next »
  6. ×

    To search the entire text of this book, type in your search term here and press Enter.

    « Back Next »
  7. ×

    Share a link to this book page on your preferred social network or via email.

    « Back Next »
  8. ×

    View our suggested citation for this chapter.

    « Back Next »
  9. ×

    Ready to take your reading offline? Click here to buy this book in print or download it as a free PDF, if available.

    « Back Next »
Stay Connected!