National Academies Press: OpenBook

Improving the Quality of Health Care for Mental and Substance-Use Conditions (2006)

Chapter: Appendix C: Mental and Substance Use Health for Veterans: Experience with Performance Evaluation in the Department of Veterans Affairs

« Previous: Appendix B: Contraints on Sharing Mental Health and Substance Use Treatment Information Imposed by Federal and State Medical Records Privacy Laws
Suggested Citation:"Appendix C: Mental and Substance Use Health for Veterans: Experience with Performance Evaluation in the Department of Veterans Affairs." Institute of Medicine. 2006. Improving the Quality of Health Care for Mental and Substance-Use Conditions. Washington, DC: The National Academies Press. doi: 10.17226/11470.
×

Appendix C
Mental and Substance-Use Health Services for Veterans: Experience with Performance Evaluation in the Department of Veterans Affairs

Robert Rosenheck, MD

Director, VA Northeast Program Evaluation Center (NEPEC), West Haven, CT

Professor of Psychiatry, Public Health and at the Child Study Center Yale Medical School, New Haven, CT


August 2004


Prepared for the Institute of Medicine Committee on Crossing the Quality Chasm: Adaptation to Mental Health and Addictive Disorders


Acknowledgement: Paul Errera MD, Thomas Horvath MD, Laurent Lehmann MD, Mark Shelhorse MD, Mary Jansen PhD, Gay Koerber MA, William Van Stone MD, Robert Gresen PhD and Anthony Campinell PhD and the staff of the Strategic Healthcare Group for Mental Health in VA Central Office have provided invaluable support over many years. The staff of NEPEC is responsible for most of the work reported here (but not for the errors, which are my own), specifically project directors Mayur Desai PhD, Rani Desai PhD, Alan Fontana PhD, Greg Greenberg PhD, Wesley Kasprow PhD, Douglas Leslie PhD, Alvin Mares PhD, James McGuire PhD, Michale Neale PhD, Sandra Resnick PhD. Thanks also to Michael Sernyak MD. Special analyses of the 2001 Survey of Veterans and the Schizophrenia PORT survey for this report were completed by Greg Greenberg and Rani Desai.

Summary

As the largest integrated health and social welfare agency in the United States, the Department of Veterans Affairs is a unique and potentially informative setting in which to examine the challenges of mental health and substance use treatment services quality and performance management.

Suggested Citation:"Appendix C: Mental and Substance Use Health for Veterans: Experience with Performance Evaluation in the Department of Veterans Affairs." Institute of Medicine. 2006. Improving the Quality of Health Care for Mental and Substance-Use Conditions. Washington, DC: The National Academies Press. doi: 10.17226/11470.
×

U.S. Veterans

Of 25 million U.S. veterans, 21% used Veterans Affairs (VA) services in the past year, and 2.7% used VA mental health or substance abuse (MH/SA) services. Although all veterans are now eligible for VA services, those most likely to use VA services receive VA income benefits are older, poorer, and less likely to have health insurance.

Treatment of MH/SA in VA

In 2003, 1.2 million veterans received a MH/SA diagnosis in VA, about 25% of all VA users. While they were a diagnostically mixed, Global Assessment of Functioning scores averaged 53, suggesting poor functioning, and 19% were dually diagnosed. VA is a cabinet-level agency with many important stakeholders. Concern about war-related Post Traumatic Stress Disorder (PTSD) and homelessness among veterans have given mental health issues greater prominence in the VA community in recent years. In 1995, a major reform was initiated which closed most MH/SA inpatient beds, nearly doubled outpatients treated, and emphasized accountability and performance measurement.

Linkage of VA with the Department of Defense (DoD) and Other Mental Health, Medical, and Social Service Systems

There has been great interest recently in smoothing the transition from DoD to VA, although the integration of information systems has yet to take place. Most VA patients get all of their MH/SA and medical services from VA. Although there has been concern that with extensive recent bed closures, VA patients would be forced to seek care in other health systems and might experience an increased risk of incarceration or suicide, empirical studies conducted thus far have not shown a significant increase in these problems.

Development of MH/SA Quality Measurement and Quality Management in VA

During the past 20 years there have been two notable phases in the development of VA MH/SA services. The first was initiated by the leader of mental health programs in VA central office from 1985–1994 and involved expansion of specialized mental health programs such as Assertive Community Treatment, homeless

Suggested Citation:"Appendix C: Mental and Substance Use Health for Veterans: Experience with Performance Evaluation in the Department of Veterans Affairs." Institute of Medicine. 2006. Improving the Quality of Health Care for Mental and Substance-Use Conditions. Washington, DC: The National Academies Press. doi: 10.17226/11470.
×

outreach, and transitional employment. The second was initiated in 1995 by the undersecretary for Health and brought changes in mental health service delivery as part of a major system-wide shift from operating as a hospital-based system of care to a community or population-based system of care. In both phases quality and performance measurement were crucial tools in guiding organizational change. A third phase, characterized by system-wide focus on building MH/SA quality is evolving in response to the recent report of the President’s New Freedom Commission on Mental Health.

Quality of VA MH/SA Care

The assessment of quality requires comparison of providers with standards or benchmarks, with risk adjustment for factors that may confound these comparisons. Evaluation of the quality of MH/SA care at VA facilities has been based on comparisons with: (1) VA system average performance, (2) VA performance in prior years, (3) the performance of other systems of care, and (4) comparison of care received by minorities with the majority population. Methods of quality measurement and benchmarking in VA are demonstrated for six aims highlighted in a previous Institute of Medicine (IOM) report: safety, effectiveness, person-centeredness, timeliness, efficiency, and equitability.

Front-Line Experience

Performance management in health care is sometimes experienced ambivalently by front line managers and clinicians. While they often feel empowered by access to data and find it allows them to improve the care they provide, there is also concern that measures are imperfect; that they do not take account of differences across facilities in case mix and in available community resources; that measures can be manipulated or “gamed,” resulting in unfair comparisons; and that managerial pressure to improve performance sometimes creates an atmosphere of personal criticism more than joint problem solving.

Conclusion

The complexity and uncertainty of the health care enterprise must be managed through comprehensive quality monitoring systems used by creative and committed leaders in competent organizations. VA has embraced this challenge.

Suggested Citation:"Appendix C: Mental and Substance Use Health for Veterans: Experience with Performance Evaluation in the Department of Veterans Affairs." Institute of Medicine. 2006. Improving the Quality of Health Care for Mental and Substance-Use Conditions. Washington, DC: The National Academies Press. doi: 10.17226/11470.
×

INTRODUCTION: THE DEPARTMENT OF VETERANS AFFAIRS IN AMERICAN MENTAL HEALTH CARE

To bridge a chasm, one needs, at a minimum, a clear view of the terrain surrounding it. One can imagine building a bridge from one side of a chasm to the other, from both sides toward the middle, or even from a scaffolding erected on the floor in the center of the chasm out to both sides. But it is virtually impossible to imagine bridging a chasm if one were blocked from either viewing or accessing even one of its banks. The situation faced by those who would seek to bridge the many quality chasms in mental health care in the United States is in many respects like that of an engineer bridging a complex system of chasms with access to only one of its banks. People with serious mental illness often have needs for diverse services including psychiatric care, substance abuse care, primary and specialty medical care, and numerous social services including income supports, employment, education, and housing assistance, as well as help negotiating with the criminal justice system. And yet each of these needs is addressed by a different set of agencies at different levels of government. The advantage of this decentralized approach is that it increases local control, responsiveness, and flexibility (Peterson, 1995; Smith and Lipsky, 1993). The disadvantage is that agencies tend to compete for sources of funding, carefully guard their independence, and are often wary of sharing information on individual clients, let alone releasing systematic data on their overall operation. Mental health system engineers in America thus often find themselves trying to bridge system chasms while only being able to obtain information on the small patch of ground under their own feet.

The U.S. Department of Veterans Affairs (VA) is a notable exception to this pattern. In most areas of U.S. social or health care policy, programs are operated at state or local levels by private or nonprofit providers, and the role of the federal government is limited. In contrast, the national government takes direct responsibility for providing comprehensive, lifelong, medical and social services to Veterans of the Armed Forces. National defense is the least contested area of federal dominance in the American system of government; as a result, the federal government has been given responsibility for the health and social welfare of military personnel, both on active duty and, for an increasingly large segment of veterans, after their period of military service is over. The VA thus represents the unusual case in which one agency accepts responsibility, at the national level, for providing comprehensive long-term care for a well-defined segment of the population. Mental health care provided by the VA may thus offer a uniquely informative, if atypical, opportunity to examine mental health performance monitoring and management in the American context.

Taking a broad view, this presentation will: (1) describe the veteran population of the United States, compare the mental health needs (includ-

Suggested Citation:"Appendix C: Mental and Substance Use Health for Veterans: Experience with Performance Evaluation in the Department of Veterans Affairs." Institute of Medicine. 2006. Improving the Quality of Health Care for Mental and Substance-Use Conditions. Washington, DC: The National Academies Press. doi: 10.17226/11470.
×

ing substance use treatment needs) of veterans and those of other Americans, and compare veterans who use VA services with both nonveterans and other veterans who do not. It will then (2) present an overview of the VA health care system, the Veterans Health Administration (VHA)—and specifically its delivery of mental health and substance abuse services, paying special attention to the most prominent needs of the treated population, basic organizational structures, and notable changes in the delivery of those services in the past decade. (3) Next, data on the linkages between VA and non-VA mental health and social welfare systems will be presented to allow evaluation of the level of self-containment of VA mental health care and the nature of its linkages with other systems. The next section (4) describes the organizational processes through which quality management has been developed in VA mental health and substance abuse care in recent years. Having presented the context of mental health performance management, in the next section (5) we present evidence concerning the safety, effectiveness, person-centeredness, timeliness, efficiency and equitability of VA mental health and substance abuse care as it has changed in recent years and as it compares to other health care systems. Finally, (6) we touch on an area that has received virtually no systematic attention, the sometimes ambivalent reactions of front-line health system managers and clinicians to the implementation of performance management systems.

I. AMERICA’S VETERANS: MENTAL HEALTH AND SUBSTANCE ABUSE STATUS AND USE OF VA SERVICES

America’s Veterans

In 2001, the national Survey of Veterans (SOV) conducted detailed interviews with a nationally representative sample of 20,000 veterans identified through VA administrative records and random digit dialing (USVA, 2004). Population estimates derived from the survey were based on an overall estimate of 25,196,036 living veterans in 2001, which included 12.4% of all U.S. adults and 24.5% of men 18 years or older (U.S. Census Bureau, 2001) (see column 1 of Table 1). The two most distinctive characteristics of the veteran population is that it is overwhelmingly male (94%) and that its age distribution is shaped by defense manpower needs and particularly wartime recruitment, rather than by the natural rate of population growth.

Veterans are older than other Americans, first, because eligibility for military service begins at 18. In addition, and perhaps more important, World War II, the Korean conflict, and the Vietnam conflict, spanning 34 years from 1941 to 1975, were fought by far larger forces than have served in the 28 years since 1975. Altogether 30 million troops served during the

Suggested Citation:"Appendix C: Mental and Substance Use Health for Veterans: Experience with Performance Evaluation in the Department of Veterans Affairs." Institute of Medicine. 2006. Improving the Quality of Health Care for Mental and Substance-Use Conditions. Washington, DC: The National Academies Press. doi: 10.17226/11470.
×

three major war eras from 1941 to 1975 while only 5.7 million living veterans entered military service since 1975 (U.S. Census Bureaus, 2001). Thus 46% of veterans are 60 years old or more as compared to only 16.6% of the general population and 14.5% of men. A detailed characterization of the U.S. veteran population based on the 2001 SOV is presented in the first column of Table 1. Veterans do not differ markedly from the rest of the U.S. population in racial composition, education, or employment although there are somewhat fewer blacks and Hispanics and educational levels are somewhat higher among veterans, most likely due to the availability of specific veterans’ educational benefits.

Mental Health and Substance Abuse Disorders Among Veterans and Nonveterans in the General Population

Self-report data from the SOV show that 6.6% of veterans report having received services for a MH/SA problem in the past year, but these data do not allow comparison with the MH/SA status of the general population. The most useful study for comparing MH/SA problems of veteran and nonveteran men is a secondary analysis of data from the Epidemiological Catchment Area (ECA), which surveyed the mental health status of 18,572 Americans, including 10,954 men, and oversampled older Americans, in five locations in 1980 (Norquist et al., 1990).

ECA data reveal no differences in lifetime prevalence of mental health disorder among veterans of World War II or either the Korean or Vietnam conflict eras and age-matched nonveteran men. In contrast, veterans of the post-Vietnam era (the initial period of the All Volunteer Force [AVF]) show a greater prevalence of lifetime mental disorder (54.6% of veterans vs. 40.9% of nonveterans [p <.0001]). Data on 6-month prevalence of mental disorder show a similar pattern, although World War II era veterans had a significantly lower overall prevalence of mental disorder than nonveterans (11.8% vs. 17.7%, p <.01).

Examination of specific lifetime disorders shows that World War II era veterans had lower prevalences of any nonsubstance abuse disorder than nonveterans (12.2% vs. 18.5%, p <.01); Vietnam era veterans had lower prevalence of schizophrenic disorders (0.8% vs. 2.2%, p <.05) and affective disorders (4.4% vs. 8.3%, p <.01); and post-Vietnam veterans had higher lifetime prevalence of substance abuse disorders (47.4% vs. 30.6%, p <.01, including both alcohol and drug disorders) and antisocial personality disorder (14.9% vs. 5.8%) but lower prevalence of schizophrenic disorder than nonveterans (0.3% vs. 1.5%, p <.01).

Findings of greater rates of mental illness and especially substance abuse among veterans of the AVF are consistent with several studies showing greater substance use among military personnel in the immediate post-

Suggested Citation:"Appendix C: Mental and Substance Use Health for Veterans: Experience with Performance Evaluation in the Department of Veterans Affairs." Institute of Medicine. 2006. Improving the Quality of Health Care for Mental and Substance-Use Conditions. Washington, DC: The National Academies Press. doi: 10.17226/11470.
×

Vietnam era (Rosenheck et al., 1996a) and with studies that have demonstrated a three to four times greater risk of homelessness among post-Vietnam veterans as compared to nonveterans, as well as among female veterans (who have always served on a voluntary basis) (Gamache et al., 2001, 2003). The increased risk of homelessness is completely absent or not statistically significant among veterans of earlier eras in which the draft guaranteed a more representative military force. Thus a major issue in the psychiatric epidemiology of U.S. veterans appears to be the shift from the draft to the AVF. There have been far fewer veterans since the end of the Vietnam conflict, but they appear to have a greater risk of MH/SA problems, not because of the hazards of military service, but because of self-selection processes among those who volunteer.

Findings from the ECA are also consistent with the results of the National Vietnam Veterans Readjustment Study (NVVRS) (Kulka et al., 1990), a major epidemiologic study of representative samples of Vietnam era veterans and a matched sample of nonveterans. While the NVVRS found higher rates of posttraumatic stress disorder (PTSD) among veterans exposed to high levels of combat than among other veterans who were not and civilians; rates of other mental disorders did not differ between veteran and nonveteran populations.

A recent analysis of data from 12,480 male respondents aged 25–60 in the National Household Drug Abuse Surveys from 1994, 1997, and 1998 showed veterans reported greater rates of near-daily alcohol use in the past year (22.9 vs. 19.2%, p <.001) but lower rates of illicit drug use (10.0% vs. 12.9%, p <.001) (Tessler et al., in press). Similarly, an epidemiologic study that compared homeless veteran and nonveteran men in Los Angeles found that veterans were less likely to have nonsubstance abuse mental health disorders (47.5% vs. 65.2%, p <.01) but more likely to meet criteria for alcohol abuse or dependence (72.3% vs. 59.8%, p <.05) (Rosenheck and Koegel, 1993). Both these studies involve representative samples of the veteran population, not those involved in treatment.

MH/SA Status Among Veteran and Nonveteran Users of Mental Health Services

Three studies have compared veteran and nonveteran men who were using MH/SA services (Desai et al., in press-b; Rosenheck et al., 2000a; Tessler et al., 2002). These studies generally have found veterans to be older, less likely to be minorities, better educated, and with higher incomes, and analyses were adjusted for these differences in comparisons of mental health status.

The Schizophrenia Patient Outcomes Research Team (PORT) study of representative samples of patients treated for schizophrenia in Ohio and

Suggested Citation:"Appendix C: Mental and Substance Use Health for Veterans: Experience with Performance Evaluation in the Department of Veterans Affairs." Institute of Medicine. 2006. Improving the Quality of Health Care for Mental and Substance-Use Conditions. Washington, DC: The National Academies Press. doi: 10.17226/11470.
×

Georgia included an over-sampling of VA patients to allow adequate power for comparison of VA–non-VA male service users (N = 466 VA patients and 279 non-VA male patients) (Rosenheck et al., 2000a). After adjusting for age and race differences, no significant differences were found on measures of psychosis, depression, or substance abuse.

A comparison of 1,252 veteran and 3,236 nonveteran men treated at 18 sites in the ACCESS demonstration of service system integration for homeless people with severe mental illness also found no differences between veterans and nonveterans on psychiatric or drug problems, although veterans had somewhat more severe alcohol problems as measured by the Addiction Severity Index (Tessler et al., 2002).

Finally, the Connecticut Outcome Study compared 196 VA patients and 337 non-VA patients treated at nearby state-operated Community Mental Health Centers and also found no significant differences in measures of psychiatric symptoms or substance abuse, after adjustment for age, race and income (Desai et al., in press).

Use of VA Services

Data from the 2001 SOV show that 20.5% of veterans reported using any VA services (i.e., not specifically MH/SA services) in the past year and 34.4% in their lifetimes. These figures are substantially higher than those recorded in a similar national survey conducted in 1987. In that survey, only 5.8% reported VA service use in the past year and 21.2% lifetime (Rosenheck and Massari, 1993). These substantial changes reflect at least three factors. First, eligibility for VA services was vastly expanded in 1996 from those who receive VA compensation or pension benefits or have low incomes, estimated to have represented only 9.4 million veterans (Kizer, 1999), to the entire population of 25 million veterans. In addition, major changes in the configuration of VA facilities have made services far more accessible. In the 1990s major reductions in inpatient beds allowed expansion of outpatient care and the establishment of over 500 accessible community-based outpatient clinics (GAO, 2001). In 2003, the average veteran lived 12.2 miles from the nearest VA facility as compared to 32.0 miles in 1994 (Greenberg and Rosenheck, 2003; Rosenheck and Cicchetti, 1995). In addition, the increasing numbers of uninsured Americans and the growing cost and importance of prescription drugs have also contributed to the growing demand for virtually free VA services.

In spite of the changes in eligibility, veteran characteristics that are associated with use of VA services have changed little since 1987. Columns 2 and 3 in Table 1 show the proportions of veterans in each subgroup who used VA services in the past year and in their lifetimes. Both recent and lifetime VA service use is associated with greater age, minority status, low

Suggested Citation:"Appendix C: Mental and Substance Use Health for Veterans: Experience with Performance Evaluation in the Department of Veterans Affairs." Institute of Medicine. 2006. Improving the Quality of Health Care for Mental and Substance-Use Conditions. Washington, DC: The National Academies Press. doi: 10.17226/11470.
×

education and income, lack of private insurance, poor health, mental health service use, and receipt of VA compensation.

Table 2 presents a logistic regression analysis of factors that predict recent and lifetime VA service use to illustrate both the independent and the relative magnitude of the effects of each factor. The fourth and seventh columns rank the absolute value of the magnitude of these effects (both positive and negative) and show the strongest correlates of VA service use to be receipt of VA compensation, low income, lack of private insurance, poor health, age less than 30, African American race, Prisoner of War experience, and mental health service use and related disability. These factors are virtually the same as those identified in the 1987 SOV (Rosenheck and Massari, 1993).

Use of VA Mental Health Services

SOV data further reveal that 6.6% of all veterans used mental health services in the previous year, and 2.7% used VA mental health services (41% of those who used any mental health services) (Columns 4 and 5 of Table 1). Among mental health service users, too, those who used VA services are older, more likely to be minority group members, had less education and lower incomes, lacked private insurance, had poorer health, and received VA compensation (see also Table 3 for logistic regression analysis and ranking). It is notable that veterans who sought services for PTSD were especially likely to have used VA mental health services, replicating a finding from a previous analysis of the NVVRS data (Rosenheck and Fontana, 1995) and showing that, contrary to what was once popular belief, veterans with PTSD related to their military service do not avoid using VA mental health services.

The PORT survey of the treatment of schizophrenia in Ohio and Georgia allows further comparison, with the group of severely mentally ill veterans, of those who used VA services (N = 350) and those who used non-VA services (N = 170) (reanalysis based on data in Rosenheck et al., 2000a). Stepwise logistic regression showed veterans who used VA services to be 2.7 times more likely to be receiving VA compensation, 3.0 times more likely to be living in a supervised residence, 37% less likely to be black, and to have used fewer emergency services and have had less severe symptoms.

A study that focused on administrative data from state mental hospitals in eight states between 1984 and 1989 (Desai and Rosenheck, 2000) found that from 7 to 27% of men in these non-VA facilities were veterans as compared to 29–34% in the general male population, suggesting that veterans are less likely to use non-VA service than other men. In comparison with other state hospital patients, veterans were older, more like to have alcoholism and bipolar disorder, and perhaps of greatest interest: (a) lived

Suggested Citation:"Appendix C: Mental and Substance Use Health for Veterans: Experience with Performance Evaluation in the Department of Veterans Affairs." Institute of Medicine. 2006. Improving the Quality of Health Care for Mental and Substance-Use Conditions. Washington, DC: The National Academies Press. doi: 10.17226/11470.
×

further away from VA hospitals than nonveterans; (b) were from states with lower per capita expenditure on VA mental health care; and (c) were from states with higher per capita expenditure on state hospitals. Thus, in addition to personal characteristics, both residential remoteness from a VA facility and scant supply of VA services in relation to non-VA services increased veterans’ use of non-VA services.

A study of the proportion of veterans in each U.S. county who use VA MH/SA services similarly found distance from veterans’ residences to the nearest VA facility to be the strongest predictor of VA MH/SA services use, along with the relative local supply of VA and non-VA services (Rosenheck and Stolar, 1998). In fact, VA service use among veterans service connected for psychoses was specifically reduced in association with a high supply of state and county mental hospital resources, while VA use among nonpsychotic veterans was negatively associated with the supply of non-Federal general hospital resources. The impact of the supply and proximity of VA services has also been demonstrated in a sample of homeless mental health services users (Gamache et al., 2000).

II. TREATMENT OF MENTAL HEALTH AND SUBSTANCE ABUSE IN THE VA: PATIENTS, ADMINISTRATION, RELATIONSHIPS WITH OTHER FEDERAL AGENCIES, STAKEHOLDERS, AND CHANGES SINCE 1995

The VA is a cabinet level federal department that includes two major subdivisions that provide services to people with mental illness: (1) The VHA, which delivered health care services to approximately 5 million veterans in fiscal year (FY) 2003 at 162 medical centers and more than 850 facility and community based clinics; and (2) the Veterans Benefits Administration (VBA), which provided income benefits to over 2.5 million veterans in FY 2003 in addition to rehabilitation and educational support and housing loan guarantees (U.S. Department of Veterans Affairs, 2003). Of the 2.5 million veterans who received compensation from VBA in 2003, 481,000 received compensation for mental illness, and of these 47% used VA mental health services (Greenberg and Rosenheck, 2004a).

VA Patients Diagnosed with Mental Health and Substance Abuse Disorders

Administrative workload data from the VHA show that in FY 2003, 1,218,327 veterans (about 25% of all those who received VA health services) received a mental health or substance abuse (MH/SA) diagnosis (ICD-9 codes 290.00-312.99) during an inpatient, nursing home, residential, or outpatient encounter (Table 4).

Suggested Citation:"Appendix C: Mental and Substance Use Health for Veterans: Experience with Performance Evaluation in the Department of Veterans Affairs." Institute of Medicine. 2006. Improving the Quality of Health Care for Mental and Substance-Use Conditions. Washington, DC: The National Academies Press. doi: 10.17226/11470.
×

The most frequent MH/SA diagnoses (Table 4, numbered column 2) were dysthymia (41%), PTSD (20%), anxiety disorder (20%), and major depressive disorder (20%). Altogether 22% received a substance abuse diagnosis (17% alcohol abuse/dependence and 11% drug abuse/dependence) and 18% were dually diagnosed. Altogether, 38% received VA compensation for medical or psychiatric problems, almost four times that in the general population.

Among veterans who received a MH/SA diagnosis, 930,098 (76%) received a primary diagnosis for MH/SA (Table 4, numbered column 3), meaning that the MH/SA diagnosis was the primary focus of at least one contact during the year; and 705,209 of these received treatment in a MH/SA specialty program (numbered column 4) (76% of those who received a primary MH/SA diagnosis and 57% of those who received any primary or secondary MH/SA diagnosis). An additional 89,372 veterans received services in a specialty mental health program but did not receive a primary mental health diagnosis (not shown on table), for a total of 794,581 or 17% of all VA patients who received mental health services in a specialty clinic setting (Greenberg and Rosenheck, 2004a).

The most frequent MH/SA diagnoses among veterans treated in specialty clinics (Table 4, numbered column 5) were also dysthymia (43%), PTSD (31%), anxiety disorder (22%), and major depressive disorder (24%), with 26% receiving a substance abuse diagnosis (20% alcohol abuse/dependence and 16% drug abuse/dependence), and 18% were dually diagnosed.

Global Assessment of Functioning (GAF) scores based on a single item rating scale ranging from 0 to 100, which is a standard part of the psychiatric diagnosis, average 41.8 (s.d. = 13.1) among inpatients at the time of discharge and 53.3 (s.d. = 11.3) among outpatients (Greenberg and Rosenheck, 2004a). A GAF score of 50 is often used as a cutoff for severe mental illness. Thus although fewer than 15% of VA patients with MH/SA diagnoses have the most severe illnesses such as schizophrenia or bipolar disorder, there is considerable functional impairment among these patients. It is also noteworthy that although only 76% of veterans with a primary MH/SA diagnosis receive care in specialty clinics, 95% or more of those with the most serious illnesses (schizophrenia, major depressive disorder, bipolar disorder, or PTSD) receive care in MH/SA specialty clinics, and 45% receive VA compensation.

Annual surveys conducted from FY 1995 (Rosenheck et al., 1996c) to FY 2000 (Seibyl et al., 2001) showed that almost 30% of VA psychiatric inpatients, and almost 50% of those in inpatient substance abuse programs, had been homeless at the time of admission. Over 100,000 MH/SA outpatients are identified as homeless each year, about 12% of the total, which is most likely a substantial undercount, since coding for homelessness is not uniform in the outpatient files.

Suggested Citation:"Appendix C: Mental and Substance Use Health for Veterans: Experience with Performance Evaluation in the Department of Veterans Affairs." Institute of Medicine. 2006. Improving the Quality of Health Care for Mental and Substance-Use Conditions. Washington, DC: The National Academies Press. doi: 10.17226/11470.
×

Specialized MH/SA Programs

Perhaps the most basic approach to improving the quality of mental health care in VA, as elsewhere, has been through the establishment of specialized programs or treatment units. In the 1950s, VA established a community foster care program that represented one of the early efforts to transfer severely mentally ill patients from the hospital to the community (Linn et al., 1977). In the 1960s and 1970s, day hospitals (Linn et al., 1979) and day treatment centers were established as alternatives to hospitalization along with specialized inpatient and outpatient substance abuse programs and a transitional employment program that offered veterans the opportunity to work, first in workshop settings and subsequently at community jobs. More recently, specialized programs have been established to treat military-related PTSD; to conduct outreach and provide residential treatment for homeless veterans; to provide residential rehabilitation in community settings and to deliver specialized services to veterans with substance abuse problems. Table 5 summarizes workloads in specialized VA inpatient and outpatient MH/SA programs. While it appears that as many as 400,000 may receive treatment in specialized programs, these figures are not unduplicated counts and a substantial number of veterans are treated in more than one program. More will be said about the development, management, and monitoring of specialized VA MH/SA programs, below.

Administrative Organization

The VHA is led by the undersecretary for Health, a presidential appointee approved by the Senate. Line authority for operations devolves through the deputy undersecretary for operations and management to the directors of 21 Veterans Integrated Service Networks (VISNs), the regional unit of administration in VHA. VISN directors are responsible for supervision of the directors of each of the 152 local VA Medical Centers (VAMCs), about 130 of which operate specialty mental health programs. VISNs serve an average of 231,000 veterans per year with an average of 8,671 Full Time Employee Equivalents (FTEE) and consist of 4-8 medical centers, which each serve an average of 37,554 veterans per year with an average of 1,437 FTEE.

The lead mental health expert in VA Central Office (VACO) is the chief consultant of the Strategic Health Care Group for Mental Health, who provides staff support to the chief of Patient Care Services and through that position to the deputy undersecretary for Health. Thus, the national leader of mental health has no direct line or budgetary authority and acts as a staff advisor, several levels below top VHA leadership. The national Office of Quality and Performance (OQP) in VACO is responsible for designing national performance measures and does so with extensive input from the

Suggested Citation:"Appendix C: Mental and Substance Use Health for Veterans: Experience with Performance Evaluation in the Department of Veterans Affairs." Institute of Medicine. 2006. Improving the Quality of Health Care for Mental and Substance-Use Conditions. Washington, DC: The National Academies Press. doi: 10.17226/11470.
×

field. The director of OQP reports directly to the deputy undersecretary for Health.

While there is some variability in organization across VISNs and VAMCs, most have a mental health service line manager, most often a psychiatrist or psychologist, who is responsible for coordinating the delivery of mental health care by all the involved professions at that facility. At present, mental health service line managers do not have budgetary authority and appeal to VISN or VAMC leaders for resources in competition with the leadership of other medical specialties. Quality management and preparation for Joint Commission on Accreditation of Health Care Organizations (JCAHO) and Commission on Accreditation of Rehabilitation Facilities (CARF) accreditation are VISN and VAMC responsibilities, in which the mental health service line managers are responsible for the performance of the mental health programs.

Relationships with Other Federal Departments

Collaboration to facilitate the transition from Department of Defense (DoD) to VA care has been of growing interest in recent years. Staff of the VBA counsel military personnel about their VA benefits as they leave military service, but there has been no ongoing sharing of medical records or other information. Specific efforts are now being made to facilitate electronic information exchange between the agencies, especially in response to the concern about the new generation of veterans now returning from Iraq. The position of deputy Secretary for Health for Health Policy Coordination was created 2 years ago to lead interagency program development and to serve as the principle liaison between VA and the Department of Health and Human Services. VA participates actively in joint activities with other federal agencies, as in the Interagency Counsel on Homelessness and the President’s New Freedom Commission on Mental Health, and has conducted joint service and evaluation projects with Housing and Urban Development (Rosenheck et al., 2003a) and the Social Security Administration (Rosenheck et al., 1999b, 2000a), among others. VA differs from most other federal agencies in that it is a direct provider of services rather than a channel for funds, and thus collaboration in service delivery is uncommon but may grow with DoD.

Stakeholders

The primary external stakeholders in the operation of VA are the Congress, and especially the Veterans Affairs Committees, and the Veterans Service Organizations (e.g., the Paralyzed Veterans of America, Disabled Veterans of America, the Veterans of Foreign Wars, the American Legion,

Suggested Citation:"Appendix C: Mental and Substance Use Health for Veterans: Experience with Performance Evaluation in the Department of Veterans Affairs." Institute of Medicine. 2006. Improving the Quality of Health Care for Mental and Substance-Use Conditions. Washington, DC: The National Academies Press. doi: 10.17226/11470.
×

and Vietnam Veterans of America), which are especially active and influential. Perhaps because issues of PTSD and homelessness among veterans have been prominent in recent years, these stakeholders appear to have shown greater interest in mental health issues over the past 15 years, and particularly in the quality and funding of VA mental health care. Congress also established a committee with VHA, the undersecretary for Health’s Special Committee on Treatment of Severely Mentally Ill Veterans (the SMI committee), charged with making recommendations for of improving MH/SA care and monitoring maintenance of MH/SA capacity. The SMI committee is required to submit an annual report to the Congress.

Changes in MH/SA Service Delivery Since 1995

In 1995, Kenneth Kizer, MD MPH, was appointed undersecretary for Health and initiated an extensive reform of VHA. Kizer encouraged a shift to a population-based preventive and primary care focus rather than a hospital, specialty care focus. He vigorously promoted a reduction in inpatient service utilization and championed an expansion of outpatient treatment, in part through the development of community-based outpatient clinics, small satellite clinics located closer to where veterans lived. These goals were reinforced through a capitated system of resource allocation and by placing major emphasis on accountability through the use of performance measures (Kizer, 1999).

Although his focus was not specific to MH/SA, during these years VA mental health underwent a substantial transformation. Between 1995 and 2003, 66% of all general psychiatry inpatient beds and 96% of all inpatient substance abuse beds were closed. The number of long-term psychiatric patients, that is, those hospitalized for more than a year, declined by 81% and the number with psychiatric diagnoses on inpatient medical units declined by 93%. Inpatient length of stay dropped 43%, from an average of 27.8 days to 15.8 days, allowing more patients to use the remaining beds. As a result, the number of episodes of inpatient care declined by only 44%.

With the pressure of a capitated resource allocation system and population-based planning, the total number of mental health outpatients increased by 44.7%, or 5.6% per year, from 545,004 to 788,502. Perhaps to allow time to serve this increasing workload, the average number of annual visits per veteran declined from 15.1 to 12.8 (15%).

Specialized outpatient substance abuse (SA) services initially (FY 1995–FY 1998) followed this general trend, with 3% annual growth in the number of patients treated, but from FY 1998 to FY 2003 the number of veterans who received specialized outpatient substance abuse services declined by 19% (3.7%/year). Since there was no reduction in need of SA services in the

Suggested Citation:"Appendix C: Mental and Substance Use Health for Veterans: Experience with Performance Evaluation in the Department of Veterans Affairs." Institute of Medicine. 2006. Improving the Quality of Health Care for Mental and Substance-Use Conditions. Washington, DC: The National Academies Press. doi: 10.17226/11470.
×

veteran population (Tessler et al., 2005), this decline may have reflected an unintended decline in supply of VA SA services, which has grown more serious for five consecutive years.

III. LINKAGE OF VETERANS WITH THE DOD AND OTHER MENTAL HEALTH, MEDICAL, AND SOCIAL SERVICE SYSTEMS

As noted in the Introduction, VA is unique in American mental health care as an integrated national system providing comprehensive services to a designated population. There are, however, no restrictions that prevent veterans who use VA services from using other systems of care. In this section we examine the involvement of VA patients with other systems both to better understand the context of VA care and to assess how self-contained VA and its service users actually are.

Issues of confidentiality often complicate examination of service use across systems. While it is occasionally possible to merge data using identifiers such as social security numbers, there are two other approaches that do not require common identifiers. In the first approach, VA patients can be surveyed about their use of non-VA services. In the second, populations can be matched probabilistically on the basis of the degree of overlap in the frequency distribution of birthdates. The greater the overlap in the distribution of birthdates, the greater the likelihood of an overlap in populations (Pandiani et al., 1998).

Transition from DoD to VA

There has been only one study of the flow of mental health patients from DoD to VA. In that study (Mojtabi et al., 2003), records of patients discharged from military service for schizophrenia or bipolar or major affective disorder were merged with VA service use data. Only 52% of discharged veterans had contact with the VA system. Notably, neither women nor minorities were any less likely than other veterans to find their way to the VA. It is unknown whether those who did not contact VA had adequate access to service elsewhere or had been discouraged, somehow, from using VA services.

Cross MH/SA System Use

Several studies have examined use of non-VA MH/SA services by VA patients. First, a number of recent clinical trials have collected detailed cost data on VA and non-VA service use. Data from a study of supported housing for homeless veterans, most of whom had SA problems, found that 23% of all health costs over a three-year period were from non-VA sources

Suggested Citation:"Appendix C: Mental and Substance Use Health for Veterans: Experience with Performance Evaluation in the Department of Veterans Affairs." Institute of Medicine. 2006. Improving the Quality of Health Care for Mental and Substance-Use Conditions. Washington, DC: The National Academies Press. doi: 10.17226/11470.
×

(Rosenheck et al., 2003a). In contrast, two studies of the pharmacologic treatment of schizophrenia found less than 7% of annual health care costs were attributed to non-VA services (Rosenheck et al., 1997a, 2003d).

A study based on merged administrative data from VA and from virtually all public providers of public mental health services in Philadelphia county found that over a six year period from 1988-1993, 17% of VA MH/SA service users used any non-VA MH/SA services in at least 1 year, with 7% using non-VA services in the final year (Desai et al., 2001). In that study there were no differences in cross system use between mental health (MH), SA, and dually diagnosed veterans. Cost data from a related analysis of a subset of Philadelphia veterans suggest that only 4% of total costs among VA MH/SA users were attributable to non-VA service use.

A similar study of 10,950 VA MH/SA users in Colorado from 1995 to 1997 found only 7.7% used services of the state mental health agency over these three years, although annual rates increased from 2.9 to 5.9% over the years (Desai and Rosenheck, 2002). Veterans most likely to use non-VA services had made more extensive use of VA services, resided further away from the nearest VA facility, and lived closer to the nearest non-VA facility.

Finally, two studies examined whether VA inpatient bed closures during the 1990s resulted in greater use of non-VA mental health services. The first study, based on merged data from three large cities in Connecticut with both VA and state mental health agency facilities, found that closure of 80% of VA mental health beds in the state in 1996 resulted in a statistically significant but small increase in the proportion of VA patients who used state mental health services, from 2.7 to 3.6%, but that the proportion of total costs borne by the state ranged from 5.7 to 9.6% over the years studied (1993–1998) and did not increase significantly.

The second study (Rosenheck et al., 2000c) used population probability sampling to compare rates of admission to non-VA inpatient units in northern New York State in association with closure of 37% of VA mental health beds in the region. While finding no significant time trend, the study reported greater risk of admission among VA inpatients than outpatients and greater rates of admission to non-VA hospitals among dually diagnosed and SA patients than among mental health patients.

Primary Care and Specialty Medical Services

It might be expected that in an integrated system that provides both MH/SA and general medical services, access to medical services might be superior. However, two studies that used survey data to compare access to medical services among severely mentally ill patients in VA and non-VA MH systems failed to find any significant differences (Desai et al., in press-b; Rosenheck et al., 2000a). In addition, a randomized controlled trial that

Suggested Citation:"Appendix C: Mental and Substance Use Health for Veterans: Experience with Performance Evaluation in the Department of Veterans Affairs." Institute of Medicine. 2006. Improving the Quality of Health Care for Mental and Substance-Use Conditions. Washington, DC: The National Academies Press. doi: 10.17226/11470.
×

compared medical care quality and outcomes in a sample of severely mentally ill veterans when treated in standard VA medical clinics and in an integrated primary care clinic colocated within the mental health clinic area found that both quality and outcomes were significantly increased in the colocated clinic (Druss et al., 2001). Analysis of data on homeless veterans has also identified substantial barriers to accessing primary care services in VA (Desai et al., 2003).

On the other hand, data on quality of preventive services, diabetes care, post-MI care and health and nutrition counseling in VA show that MH/SA veterans who received at least three primary medical care visits had a quality of care that was similar to other veterans but superior to similar measures from non-VA systems (Desai et al., 2002a,b,c; Druss et al., 2002). It seems that some veterans with MH/SA problems have difficulty accessing primary care services, but those who do receive services have access to high-quality care.

Criminal Justice Involvement

There has been considerable concern, and substantial speculation, that VA bed closures have resulted in increased incarceration among former VA patients. Using the population probability matching method (Rosenheck et al., 2000d), incarceration rates among VA patients in northern New York State were not found to have increased over four years during which 37% of mental health beds were closed. Over these years, incarceration among veterans with MH problems alone ranged from 1.3 to 8.0%, as compared 12–15% among substance users, and 8–16% among dually diagnosed veterans. The overall incarceration rate among VA MH/SA patients of 11.6% was quadruple that of the general population (2.5%) but less than that found among general hospital patients (23%) or state hospital populations in northern New York (22%), in part, because veterans were older.

Some VA homeless outreach programs have undertaken active outreach to veterans with MH/SA problems in jails. A study of one such program in Los Angeles (McGuire et al., 2003), showed veterans in the L.A. jail to have substantial health problems, most prominently with drug abuse. Outreach to these veterans did not result in substantially increased service use or costs as compared to outreach to other homeless veterans, although the benefit of this type of intervention has not been evaluated.

Collaborative Relationships with Other Agencies

There has been substantial emphasis on integrating systems of care, especially for homeless people with mental illness. In 1993, VA initiated the Community Homelessness Assessment, Local Education, and Networking

Suggested Citation:"Appendix C: Mental and Substance Use Health for Veterans: Experience with Performance Evaluation in the Department of Veterans Affairs." Institute of Medicine. 2006. Improving the Quality of Health Care for Mental and Substance-Use Conditions. Washington, DC: The National Academies Press. doi: 10.17226/11470.
×

Groups (CHALENG) process at each medical center. Through CHALENG, all agencies concerned with services for homeless veterans are invited to meet at the local VAMC to review the unmet needs of homeless veterans and to plan collaborative interventions to address those needs. Analysis of data gathered at these meetings has suggested that interorganizational relationships are strongest where VA has invested funds in contracts with non-VA providers (McGuire et al., 2002).

As noted previously, VA has conducted successful demonstration projects with demonstrable benefits to veterans in housing and quality of life, in collaboration with HUD (Rosenheck et al., 2003a) and the Social Security Administration (Rosenheck et al., 1999b, 2000b). These specifically targeted interventions stand in notable contrast to more global efforts at organizational integration, which have not demonstrated benefits to clients even though they brought about changes in intraorganizational interactions (Goldman et al., 2002; Rosenheck et al., 2002).

IV. DEVELOPMENT OF MH/SA QUALITY MEASUREMENT AND QUALITY MANAGEMENT IN VA

Discussions of quality and/or outcomes improvement not uncommonly focus on the problem of real-world measurement. Such discussions often seem to assume that the numbers cannot only speak for themselves, but can also change the behavior of people whose efforts they reflect. Performance data, however, have little meaning or usefulness when taken out of their organizational context (Rosenheck, 2001a,b), and any meaningful quality monitoring effort must have a manager or, more abstractly, an agent who wants to use the data to accomplish some goal or goals. In addition, the agent must (1) have adequate authority and must be able to (2) identify appropriate target audiences, (3) communicate with those audiences, (4) generate credibility and legitimacy for the enterprise, and (5) have access to adequate analytic capacity—in short, the agent needs a well-functioning organization.

It is also important to recognize that one cannot assume that the goal of performance measurement is the improvement of performance. In a famous paper, sociologists Meyer and Rowan (1977) pointed out that while the schools and school administrators they studied collected immense amounts of data from their students in the form of test scores, they rarely used those scores to change their educational methods. More generally, they concluded, organizational activities are often maintained less because of their goal-furthering functions than because they become institutionalized, self-legitimating activities in and of themselves. Organizational actors, thus, often do things merely because they are “the thing to do.” In the absence of

Suggested Citation:"Appendix C: Mental and Substance Use Health for Veterans: Experience with Performance Evaluation in the Department of Veterans Affairs." Institute of Medicine. 2006. Improving the Quality of Health Care for Mental and Substance-Use Conditions. Washington, DC: The National Academies Press. doi: 10.17226/11470.
×

leadership committed to change and development, MH/SA performance data may serve this more limited legitimizing function.

VA is a large and complex organization, with almost 200,000 employees and at least 9 hierarchical levels separating the undersecretary for Health from the veteran. There are active agents concerned about managing quality of care at every level of every organization. Once the bridging of quality chasms is understood as an organizational process, it becomes clearer that it is impossible for any individual to comprehensively grasp the MH/SA quality improvement activities of VA in its entirety. We live in a world that is driven increasingly by networks rather than hierarchies (Castells, 2000), and this account will no doubt overemphasize initiatives in which I have been involved. The goal is thus not to present a comprehensive view of MH/SA quality management in VA, but to present one broad national perspective that will identify informative experiences and perspectives.

Evaluation and Monitoring of Specialized VA MH/SA Programs

As noted above, one of the basic approaches to improving the quality of MH care in VA, as elsewhere, has been through the establishment of specialized programs or treatment units. However, while funds for these programs have typically been distributed from VA Central Office in Washington, through the mid-1980s, there was no systematic monitoring of program performance other than mandatory workload reporting, although some specialized programs had been evaluated by VA researchers (Linn et al., 1977, 1979).

In 1985, Paul Errera, MD, professor of psychiatry at Yale and chief of psychiatry at the West Haven VAMC, was appointed chief of what was then called the Mental Health and Behavioral Sciences Service in VACO (Errera, 1988) (full disclosure: I have worked closely with Dr. Errera for the past 30 years and was involved in the evaluation of many of his initiatives). While applying for this position he read a book on program implementation by two Berkeley political scientists (Pressman and Wildavsky, 1971). Progress in government, they argued, was only possible if plausible initiatives were not assumed to be effective, but rather were taken as learning opportunities, to be evaluated with the tools of science. Errera took this exhortation to heart and in the final negotiations obtained agreement that he would use his Yale colleagues to evaluate new programs he might initiate. The VA’s Northeast Program Evaluation Center (NEPEC) grew out of these evaluations.

Errera had two objectives: first, to expand the capacity of VA to deliver community-based MH/SA care, and second, to prevent what he saw as the steady erosion of MH/SA resources (Tomich, 1992). In his experience at a

Suggested Citation:"Appendix C: Mental and Substance Use Health for Veterans: Experience with Performance Evaluation in the Department of Veterans Affairs." Institute of Medicine. 2006. Improving the Quality of Health Care for Mental and Substance-Use Conditions. Washington, DC: The National Academies Press. doi: 10.17226/11470.
×

university-affiliated VAMC, resources targeted at MH/SA programs often were diverted to more prestigious medical specialties, and he had been frustrated at his inability to staunch the loss of resources for a highly stigmatized and politically weak group of veterans. His strategy was incremental and opportunistic—he sought any small step forward where an opportunity arose. He also realized that most medical experts thought psychiatry lacked a scientific base and that its outcomes could not be measured. Performance data, he reasoned, could increase the credibility and legitimacy of his initiatives, supporting their preservation and expansion. In part because of the new availability of desk-top computers (the West Haven VA had purchased it first Apple II-E computer the year before Errera went to Washington), it had become possible to monitor the performance and clinical outcomes of hundreds of programs relatively cheaply and flexibly.

The strategy was effective in one additional, unexpected way. Implementing new programs with built-in performance monitoring systems turned out to be a useful approach to training because it clearly communicated to staff and supervisors what the expectations were for both treatment process and outcome. It was also effective in winning legitimacy for the programs in VA and in the Congress, which began to require annual reports on newly funded initiatives. During Errera’s nine years in Washington, Congress funded hundreds of new programs for Vietnam veterans with PTSD; outreach and residential treatment for homeless veterans; both inpatient and outpatient substance abuse treatment; and community-oriented work restoration programs. In partnership with one of VA’s Regional Directors, he initiated a 10-site pilot program of Assertive Community Treatment, following the model developed by Stein and Test in Wisconsin (Stein and Test, 1980). Ironically, it was not Errera but the Regional Director who wanted and funded an experimental cost-effectiveness evaluation. The evaluation study showed the approach to be cost-effective in the VA setting (Rosenheck and Neale, 1998), and it was expanded during Errera’s tenure to 30 sites, and currently operates at almost 80 sites (Neale et al., 2003). Performance data on all of these programs are provided to each site in quarterly installments with a comprehensive Annual Report. Front-line staff find these reports useful in communicating the nature of their activities and accomplishments (most of which take place out of institutional sight in community settings) to their local leadership.

In addition, most of the programs are initiated through training conferences that focus on both clinical concepts and evaluation procedures. These are followed by monthly telephone conference calls at which emergent issues are discussed and clinical experiences are shared. Even with 50 or more program sites on the line, conversational engagement has been achievable, especially when many of the participants have met each other face-to-face at the training conferences. Evaluation reports are reviewed on these

Suggested Citation:"Appendix C: Mental and Substance Use Health for Veterans: Experience with Performance Evaluation in the Department of Veterans Affairs." Institute of Medicine. 2006. Improving the Quality of Health Care for Mental and Substance-Use Conditions. Washington, DC: The National Academies Press. doi: 10.17226/11470.
×

calls providing both statistical and administrative guidance to clinicians in how to make use of the data. When a national Outlook-based VA intranet e-mail system was established, conference call communication was supplemented by continuous program-wide e-mail chatter. Through this process participants join in the formation of a nonhierarchical learning community through which local experiences and lessons learned can be widely shared. All reports identify sites by name so that those who were performing poorly could identify and learn from those who were performing well.

While Errera’s initiatives were positively received, the “fencing” of funds for new MH/SA programs (i.e., diversion to other uses was prohibited) was experienced as an undue constraint by managers responsible for the full range of VA medical programs, MH/SA and otherwise (Tomich, 1992). After 7 years, fencing was eliminated, with some resultant staff losses and program closures at some sites. Although performance monitoring and management of clinical practice runs counter to norms of professional autonomy, it seems to be accepted. Constraints on the funding decisions of local managers, however, especially over extended periods of time, have been far less acceptable.

In addition, some MH/SA managers complained that the new programs sometimes distanced themselves from other local MH/SA programs because of their special national involvement and followed national practice models too rigidly, as “stovepipes,” “chimneys,” or “silos.” The conflict between adherence to evidence-based practice standards and local flexibility seems to be an intrinsic feature of centrally guided dissemination.

Perhaps the principal lesson of these experiences is that meaningful and effective performance measurement and management are most likely to occur in a well-developed, goal-directed organizational context. The numbers do not speak for themselves.

Expansion of Quality Management in the “New VA” after 1995

When Kenneth W. Kizer, MD, MPH, took over as VA undersecretary for Health in 1995, he considered the future of the agency to be in jeopardy. In response, he initiated what he described as “the most radical redesign since the system was created in 1948.” (Kizer, 1999:3), with the overall goal of increasing health care value, defined as quality of care per dollar spent—the return to the taxpayer. He laid out his values, principals and specific plans in explicit detail in a number of widely circulated reports (Kizer, 1995, 1996; Kizer and Garthwaite, 1997) and described a first phase of operational transformation and second phase of quality transformation.

In the first phase, VHA was reorganized into Veterans Integrated Service Networks “premised on funding care for populations rather than

Suggested Citation:"Appendix C: Mental and Substance Use Health for Veterans: Experience with Performance Evaluation in the Department of Veterans Affairs." Institute of Medicine. 2006. Improving the Quality of Health Care for Mental and Substance-Use Conditions. Washington, DC: The National Academies Press. doi: 10.17226/11470.
×

facilities, with a concomitant shift in the primary focus of care from hospitals to ambulatory and community-based settings” (Kizer, 1999:6). The restructuring of VHA was thus designed to change basic values and orientations—and it did. Between 1994 and 1998, 52% of all VHA beds were closed, the proportion of patients enrolled in primary care went from 10 to 80%, the proportion of outpatient surgeries increased from 35 to 75%; and 216 community-based outpatient clinics were established to improve access. Kizer also promoted the passage of legislation that would expand eligibility for VA services to all veterans, a change that, by FY 2002, would result in vast increases in enrollment, especially among older veterans seeking low-cost prescription drugs that were not available from Medicare.

In the second phase, in addition to these structural changes, an extensive program of national performance measurement was initiated, which systematically assessed patient satisfaction and which resulted in documented improvements in standard indicators reflecting, among other things, the delivery of preventive primary care (including screening for depression), care of chronic disease, and palliative care (Jha et al., 2003). VA also exceeded Medicare on most measures, but data on rates of screening for depression were not available from Medicare for comparison. Data reports were circulated widely, with detailed information on the performance of each VISN and VAMC. The goals for this effort were both internal and external: (1) to improve the quality of care and establish an overall culture of accountability and quality improvement in VHA, and (2) to demonstrate to the taxpaying public that VA health care was a good investment.

Kizer also developed an overall management strategy for fostering quality improvement that consisted of dual systems of: (1) central regulation and (2) competition and rewards. He developed a personal performance contract with each VISN director each year and encouraged them to do likewise with their subordinates. He also established systems of bonuses for high-performance leaders and awards for exceptional accomplishment.

Although MH/SA care was not one of the principal areas of Kizer’s initial attention, it was profoundly affected by the emphasis on reducing inpatient care, with substantial reductions in bed capacity. While reductions in general psychiatry inpatient beds were similar to those in non-MH specialties, almost all of the SA inpatient beds were closed. To provide alternative care, residential rehabilitation and domiciliary programs were expanded, and residential treatment for homeless veterans was purchased through contracts with local providers. While Errera’s initiatives had been limited, for the most part, to establishing special programs on the periphery of the VA MH/SA system, Kizer’s reforms were far more extensive and affected the core.

Suggested Citation:"Appendix C: Mental and Substance Use Health for Veterans: Experience with Performance Evaluation in the Department of Veterans Affairs." Institute of Medicine. 2006. Improving the Quality of Health Care for Mental and Substance-Use Conditions. Washington, DC: The National Academies Press. doi: 10.17226/11470.
×

The new emphasis on local decision making and primary care seemed, to some, to pose a threat to the centrally designed and monitored programs Errera had fostered. In response to such expressions of concern, Kizer issued a directive (VHA Directive 96-051), which established a series of monitors that addressed both the performance of these special emphasis programs as well as service delivery to the broader population they were designed to serve. He also issued a directive (VHA Directive 99-030) stating that no MH/SA program could be substantially altered without approval from VACO—a directive that was inconsistent with the emphasis on local decision making and that was variably adhered to. Similarly, Congress, out of concern that extensive changes might sweep away the special programs it had funded, passed legislation (P.L. 104-262 section 104) requiring VA to maintain its capacity to provide specialized treatment within distinct programs or facilities to disabled veterans with mental illness and several other conditions. VA constructed a definition of capacity with the limited available data that addressed both the number of patients seen and total MH dollars spent on their care. This definition has proved controversial (Mulligan, 2002).

Among his initial proposals in Vision for Change (Kizer, 1995), Kizer mandated the development of a National Mental Health Program Performance Monitoring System to be developed by the Northeast Program Evaluation Center, which has continued to monitor the programs begun during Errera’s tenure. The “VA mental health report card” (Rosenheck and Cicchetti, 1995; Greenberg and Rosenheck, 2004a) addresses all VA inpatient and outpatient service delivery and, in concept, also includes the reports on the special programs. Many of the performance measures developed initially for these specialized programs have been incorporated into the national performance measurement system as it has evolved under the leadership of the OQP in recent years.

A third phase, characterized by a system-wide emphasis on building MH quality, is emerging in response to the recent report of the President’s New Freedom Commission on Mental Health (President’s New Freedom Commission on Mental Health, 2003). The deputy secretary for Health for Health Policy Coordination was actively involved in work of the Commission and a national action agenda for implementing the New Freedom commission recommendations has been developed. This document will be the foundation of a national strategic planning initiative for VA MH/SA care that has involved SMI Committee members among others.

Meyer and Rowan describe institutions, at one extreme, in which activities akin to quality management in health care become ritualized formalities with little actual impact on productive behavior. Errera and Kizer represent a quite different pole as leaders with broad-ranging agendas for change for whom quantitative quality management was one of many tools

Suggested Citation:"Appendix C: Mental and Substance Use Health for Veterans: Experience with Performance Evaluation in the Department of Veterans Affairs." Institute of Medicine. 2006. Improving the Quality of Health Care for Mental and Substance-Use Conditions. Washington, DC: The National Academies Press. doi: 10.17226/11470.
×

to be used to realize goals that are simultaneously professional (improving patient health) and more broadly political (changing organizational power configurations and values). We began this discussion with the observation that performance information may have little meaning when extracted from its organizational context. We conclude by pushing this thought further in suggesting that such information may play its most important role when used in service of a broad agenda for change. Health care organizations, with their extensive reliance on autonomous professional employees, have never operated as simple top-down hierarchies. Leaders who make the most effective use of performance data are able to facilitate the development of learning communities of peers who can enhance and support each other in pursuit of innovative professional and organizational objectives. Not only do the numbers not speak for themselves, they speak most forcefully when put to the most ambitious of purposes.

V. QUALITY OF VA MH/SA CARE

Even reliable and valid performance measures form an inadequate basis for action in the absence of a standard for comparison. To answer the question, “Is this care good enough?” requires either an absolute criterion of acceptability, based on expert consensus, or a salient reference condition for comparison. Even when there is expert consensus on a standard of practice, 100% compliance is usually not justified, since in clinical practice there are almost always clinical exceptions (Walter et al., 2004). Benchmarking to the actual performance of a relevant comparison program is often the most credible approach. Four types of benchmark that have commonly been used in the evaluation of VA MH/SA programs are:

  • The system average or median, based on the premise that every program should be able to achieve the current average or some standard above or below it (such as one standard deviation).

  • A historical standard, on the assumption that even if we don’t know what the right level of performance should be, deterioration in quality is unjustifiable.

  • The performance of some other system of care, on the premise that VA care should be equal to that of other systems.

  • A majority subgroup, on the assumption that ethnocultural minorities should have equitable care in comparison to the majority subgroup.

A major problem with each of these comparisons is that some of the measured differences in performance may be attributable to uncontrollable differences in either the nature of the populations being compared or to other constraints that are beyond the control of caregivers or health system

Suggested Citation:"Appendix C: Mental and Substance Use Health for Veterans: Experience with Performance Evaluation in the Department of Veterans Affairs." Institute of Medicine. 2006. Improving the Quality of Health Care for Mental and Substance-Use Conditions. Washington, DC: The National Academies Press. doi: 10.17226/11470.
×

managers. Numerous studies of VA MH/SA care have shown that risk adjustment can substantially alter the evaluation of performance of clinical units (Busch et al., 2004; Fontana et al., 2003; Greenberg and Rosenheck, 2004a; Hoff et al., 1998; Rosenheck and Cicchetti, 1998; Rosenheck and Stolar, 1998; Rosenheck et al., 1997b; Sernyak and Rosenheck, 2003; Weissman et al., 2002). While statistical methods for evening the playing field and ensuring fair comparison will not be reviewed here, their importance cannot be overemphasized, and perfect risk adjustment is unlikely to be attainable (Walter et al., 2004). Whether “good enough” risk adjustment can be attained probably varies from case to case.

The remaining sections present examples of VA MH/SA measures that address the six aims for health system improvement identified in Crossing the Quality Chasm (IOM, 2001), that is, the safety, effectiveness, person-centeredness, timeliness, efficiency, and equitability of VA MH/SA care, using each of the four benchmarks. The Appendix presents a more comprehensive set of program-specific measures used at the Northeast Program Evaluation Center to monitor and evaluate VA MH/SA care.

Safety

Suicide is perhaps the most serious safety risk in MH/SA care and is the eighth leading cause of death among men aged 45-64 nationally in the United States, with 22.4 deaths per 100,000 annually (U.S. Census Bureau, 2001). Suicide is difficult to monitor because it is, fortunately, a rare event and most health care systems do not treat enough patients to make stable or accurate estimates of the suicide rate among their patients. As an integrated national system, VA is among the few large systems of care in the United States in which suicide rates can be meaningfully measured and compared across facilities (Desai et al., 2005). Using mortality records from the National Death Index, all suicides that occurred within 12 month of discharge among 121,000 veterans discharged from VA MH inpatient units between 1994 and 1998 were identified. The suicide rate in this severely ill sample was 445/100,000, (increasing over the years but not significantly), and 13.5% of VAMCs had rates significantly higher than the average across all facilities, after risk adjustment. Time trends were also examined to see if postdischarge suicide rates had increased in association with bed closures or shortened lengths of stay. While there was a trend toward increased suicide in recent years, it was not statistically significant. Although it was impossible to externally benchmark the VA inpatient suicide rate because no comparable data are available from other institutions, it is notable that African Americans had significantly lower rates of suicide than whites. An effort was made to examine the relationship of quality indicators, especially continuity of care, and suicide. While individual patients with poor conti-

Suggested Citation:"Appendix C: Mental and Substance Use Health for Veterans: Experience with Performance Evaluation in the Department of Veterans Affairs." Institute of Medicine. 2006. Improving the Quality of Health Care for Mental and Substance-Use Conditions. Washington, DC: The National Academies Press. doi: 10.17226/11470.
×

nuity of care were at greater risk for subsequent suicide, lower continuity of care measured at the facility level was not. Risk adjustment is especially important in comparisons of suicide rates across facilities because risk varies substantially across both sociodemographic and diagnostic characteristics and even with indicators of community social capital (i.e., civic participation, trust, and cooperation) measured at the county level.

Further analyses focusing on gun suicide, an especially important issue among veterans, found that veterans living in states with lower rates of gun ownership, more restrictive gun laws, and higher social capital were less likely to commit suicide with a firearm (Desai et al., in press). Inter-facility comparisons would clearly need to adjust risk for these environmental factors since they are far beyond managerial control.

In a major study of mortality rates among VA patients during the later 1990s, Ashton et al. (2003) found no increase in mortality for several chronic conditions in VA including schizophrenia and major depressive disorder in spite of dramatically reduced inpatient utilization. However, all-cause mortality is an imprecise measure of the quality of MH care.

Another safety issue that has been systematically monitored in VA is the use of excessive doses of antipsychotic medication and antipsychotic polypharmacy, both of which pose increased risk of side effects. An annual report on antipsychotic pharmacotherapy in VA presents risk-adjusted comparison data on each VA facility (Leslie and Rosenheck, 2003a), and comparisons with data from the MarketScan® data base (a compilation of claims from private insurance plans) have shown prescription quality in VA to be similar to that among privately insured patients treated for schizophrenia (Leslie and Rosenheck, 2003b).

Because of the VA’s large data bases, its administrators have an unusual capacity to rapidly evaluate safety risks associated with newer medications. VA studies were among the first to demonstrate an increased risk of diabetes with atypical antipsychotics (Sernyak et al., 2002, 2003) and have also been the first to quantify the attributable risk of incident diabetes due to atypicals (about 0.54%) and the additional cost (about $11 per treated patient per year) (Leslie and Rosenheck, 2005).

Effectiveness

Effective care is care that improves health status and health-related quality of life. The effectiveness of treatments is most rigorously determined through formal research, especially randomized clinical trials, but also through clinical experience in the case of treatments that have yet to be studied and direct measurement of outcomes. Since randomized clinical trials and even observational outcome studies cannot be used to evaluate real-world practice in an ongoing way (because of both prohibitive cost and

Suggested Citation:"Appendix C: Mental and Substance Use Health for Veterans: Experience with Performance Evaluation in the Department of Veterans Affairs." Institute of Medicine. 2006. Improving the Quality of Health Care for Mental and Substance-Use Conditions. Washington, DC: The National Academies Press. doi: 10.17226/11470.
×

unavailability of control groups), the effectiveness of routine care is typically assessed either by determining whether evidence-based practices are in use and/or by directly measuring outcomes (see, for example, the definition of effectiveness in IOM, 2001). Pharmacologic evidence-based practices that have been monitored across the VA system include use of atypical antipsychotics (Rosenheck et al., 2001a) (see Appendix: section VI, p. 482); antidepressants in both depression (Busch et al., 2004) and alcoholism (Petrakis et al., 2003a); naltrexone in alcoholism (Petrakis et al., 2003b) and methadone in heroin addiction (Rosenheck et al., 2003c). Use of atypicals in schizophrenia and antidepressants in alcoholism is quite extensive, but only 1% of veterans with alcoholism receive naltrexone (in spite of several positive clinical trials, although a large VA trial found no benefit [Krystal et al., 2001]).

Two further studies examined whether more expensive medications (atypical antipsychotics and methadone) were less accessible at fiscally strained facilities and found that they were not (Leslie and Rosenheck, 2001; Rosenheck et al., 2003c). However, in the case of atypical antipsychotics, patients at more fiscally strained facilities were more likely to be prescribed less expensive atypicals. Psychosocial interventions that are used in VA and whose fidelity to evidence based-practices is carefully monitored include Assertive Community Treatment (ACT) (called Mental Health Intensive Case Management in VA) (Neale et al., 2003); Supported Housing (Kasprow et al., 2004); Transitional Employment (Seibyl et al., 2003), and Supported Employment (Rosenheck et al., 2003b). Adherence is somewhat variable but performance is poorest on resource- sensitive measures such as staff:patient ratios.

Many MH/SA interventions and programs have not been subject to rigorous evaluation, and operational criteria for implementing effective practices have yet to be developed. For example, while clinical practice guidelines have been developed within VA for treatment of PTSD, they are broadly worded and not subject to empirical fidelity assessment. Further, while there is broad agreement on the need for case management and residential treatment for homeless people with mental illness (especially if direct placement in permanent housing is not available), operational guidelines for the duration and intensity of such services have not been developed. In addition, in managing costly programs like ACT, administrators may not be satisfied with information showing high levels of fidelity to evidence-based models. In these cases, direct outcome assessments may be needed that demonstrate clinical improvement using psychometrically sound instruments.

There are three substantial challenges to implementing real-world outcomes monitoring. First of all, it is costly, especially if data are to be collected by independent evaluators. Second, since appropriate comparison

Suggested Citation:"Appendix C: Mental and Substance Use Health for Veterans: Experience with Performance Evaluation in the Department of Veterans Affairs." Institute of Medicine. 2006. Improving the Quality of Health Care for Mental and Substance-Use Conditions. Washington, DC: The National Academies Press. doi: 10.17226/11470.
×

groups are hard to identify, it is difficult to differentiate improvement that is attributable to effective intervention from improvement that reflects the natural waxing and waning of chronic illness. Third, assessment biases may be introduced if clinicians make ratings on their own patients or programs, and even when patients complete self-report questionnaires, since they may know that the data will be used to evaluate programs that they have come to depend on. As a result of the high cost of high quality data, to paraphrase Abraham Lincoln, “You can get some of the data on all of the people, and all of the data on some of the people, but you can’t get all of the data on all of the people.”

In VA, outcome assessment of mental health programs is selectively directed at programs that treat high-cost, high-risk patients, for example, those that receive ACT services or intensive PTSD services. Symptom and functional improvements among the more than 3,000 veterans participating in ACT in VA each year have generally improved over the years, and annual reports detail a broad array of outcome measures for each program for each year (Neale et al., 2003) (see Appendix to this paper, section III, p. 479). Measures address symptoms, quality of life, capability for self-care, employment, housing, substance use, inpatient utilization, and satisfaction with services. A system-wide outcomes monitoring evaluation of VA SA treatment similarly showed that veterans who received specialized treatment and more intensive treatment had better outcomes than those who received treatment in general medical or MH clinics (Moos et al., 2000).

PTSD outcome data have proved especially useful in addressing controversies about changes in the intensity of VA care. A study completed in the mid-1990s showed that long-term, intensive inpatient PTSD treatment was no more effective than short-term treatment but cost $18,000 more per patient/year (Fontana and Rosenheck, 1997a). In part, as a result of this study, but also because of wider changes in VA, many long-term inpatient programs were transformed into less costly short-term inpatient or halfway house programs. Stakeholders expressed concern about deterioration in the quality of VA PTSD treatment. To address these concerns, outcome monitoring data from over 6,000 episodes of care between 1993 and 2000 were reanalyzed and showed that outcomes had not changed significantly for PTSD symptoms, SA, violent behavior, and employment (Rosenheck and Fontana, 2001) and that the maintenance of effectiveness applied specifically to blacks and Hispanics as well as to whites (Rosenheck and Fontana, 2002).

The monitoring of outcomes of residential treatment for homeless veterans poses a bigger challenge, since over 10,000 episodes of residential treatment are provided each year, through several different programs, at over 100 VA medical centers. Because of the magnitude and complexity of these services, a simpler system is used to document outcomes—a standard-

Suggested Citation:"Appendix C: Mental and Substance Use Health for Veterans: Experience with Performance Evaluation in the Department of Veterans Affairs." Institute of Medicine. 2006. Improving the Quality of Health Care for Mental and Substance-Use Conditions. Washington, DC: The National Academies Press. doi: 10.17226/11470.
×

ized discharge summary records objective outcomes such as housing (40% are independently housed) and employment status (about 35% in competitive employment) at the time of discharge, as well as linkage with aftercare services, and more subjective (and thus less reliably measured) outcomes such as clinical improvement. These data are risk-adjusted using baseline information obtained at intake, and site-specific reports are circulated quarterly and summarized in an annual report to Congress.

On the largest scale, VA instituted systematic documentation of GAF ratings for all MH/SA patients in 1999. While the reliability and validity of these ratings is uncertain, annual analyses of outcome data for over 250,000 veterans have shown plausible discriminant validity and highly consistent results across medical centers over a 3-year period. The largest improvement over the six month evaluation period is observed for discharged inpatients (6.5 GAF points, s.d. = 14.4) followed by newly admitted outpatients (1.4 GAF points, s.d. = 8.7). Long-term outpatients show little improvement (0.4 GAF points, s.d. = 8.6) (Greenberg and Rosenheck, 2004a).

Three studies have collected virtually the same outcome data from patients treated in VA and in non-VA systems, allowing benchmarking of VA effectiveness against that of other systems. Because the Center for Mental Health Services’ ACCESS demonstration, which served homeless people with severe mental illness, was conducted by VA’s Northeast Program Evaluation Center, measures were similar to those used in studies of VA homeless programs, thus facilitating comparison of 8–12 month outcomes. These outcomes (addressing psychiatric symptoms, SA, housing, employment, and receipt of benefits, among others) were similar in most domains (Kasprow et al., 2002). Two other studies, the Schizophrenia Care and Outcomes Program and the Connecticut Outcome Study, traced outcomes in symptoms, community adjustment, and medication side effects among severely mentally ill patients treated at the VA Connecticut Healthcare system, and in local community mental health centers. VA and non-VA outcomes and were not substantially different.

Patient-Centered Care

The dimensions of patient-centered care described in Crossing the Quality Chasm (IOM, 2001) are well-represented on the inpatient satisfaction measure developed by the Picker Institute, which has been used in VHA since 1995. Subscales address coordination, information, timeliness, courtesy, emotional support, responsiveness to preferences, family contact, physical comfort, and transition to home and are highly intercorrelated, with bivariate coefficients ranging from 0.49 to 0.77 (Greenberg and Rosenheck, 2004a). Absolute satisfaction levels on 0–1 subscales scales,

Suggested Citation:"Appendix C: Mental and Substance Use Health for Veterans: Experience with Performance Evaluation in the Department of Veterans Affairs." Institute of Medicine. 2006. Improving the Quality of Health Care for Mental and Substance-Use Conditions. Washington, DC: The National Academies Press. doi: 10.17226/11470.
×

where 0 indicates no satisfaction and 1 indicates complete satisfaction, range from 0.56 for family involvement to 0.75 for respect for personal preferences, with few significant differences across VISNs or VAMCs (Greenberg and Rosenheck, 2004a). Subjective satisfaction and the responsiveness of VA MH/SA care has been the subject of intensive scrutiny since Dr. Kizer initiated systematic satisfaction screening in the VA (Druss and Rosenheck, 1999; Hoff et al., 1998; Kasprow et al., 1999; Rosenheck et al., 1997b). A number of studies have demonstrated that levels of satisfaction bear little relationship to outcomes and seem to be more strongly influenced by the process and intensity of care than by its effectiveness (Fontana and Rosenheck, 2001; Fontana et al., 2003). Risk adjustment is especially important and virtually reversed the results of one study of six-year time trends in satisfaction with VA inpatient MH care (Greenberg and Rosenheck, 2004b).

Recovery orientation has received increasing emphasis in MH care in recent years and can be characterized by general satisfaction with life, hopefulness, knowledge about MH care, and empowerment. A comparison of VA and non-VA patients using PORT data found no differences on these measures except that VA patients felt they knew less about MH care (Resnick et al., 2004a). Peer education groups for mental illness, another emphasis of the recovery movement in MH, have also begun to take hold in the VA setting (Resnick et al., 2004b). Alcoholics Anonymous meetings have been held on VA campuses for many decades.

Timeliness

Although detailed data on waiting times for VA MH/SA care are not available, extensive information is available on contact with outpatient MH/SA services within 30 days of discharge from an inpatient stay. This measure, originally developed for Health Employer Data Information Set (HEDIS) (HEDIS, 1999) was implemented as a national performance measure by the OQP in 1998. Between 1998 and 2003, the percentage of veterans meeting this standard increased from 72 to 77%, well above 61% among Medicare patients and even exceeding the HEDIS performance of 74% from managed care companies. A specific comparison of VA and private sector MarketScan® data showed that VA patients were 10% less likely to receive care within 30 days of discharge but only 2% less likely to have received care by 180 days. A measure of overall continuity of care favored VA, and readmission rates were similar (Leslie and Rosenheck, 2000).

The Mental Health Report Card (Greenberg and Rosenheck, 2004a) presents risk-adjusted data on these measures for each VAMC and VISN each year and also addresses access to general medical care in the 30 days

Suggested Citation:"Appendix C: Mental and Substance Use Health for Veterans: Experience with Performance Evaluation in the Department of Veterans Affairs." Institute of Medicine. 2006. Improving the Quality of Health Care for Mental and Substance-Use Conditions. Washington, DC: The National Academies Press. doi: 10.17226/11470.
×

following discharge among those with medical comorbidities (median VISN = 85 % range = 80–90%). While the HEDIS measure has been widely used in VA, and has substantial face-value, efforts to demonstrate its relationship to improved outcomes on clinical measures have shown weak and mixed results (Greenberg et al., 2002, 2003; Rosenheck et al., 1999a).

Efficiency

Dramatic reductions in reliance on inpatient care have yielded substantial efficiencies in VA MH/SA care. Even without adjustment for inflation, per capita costs for all inpatient and outpatient mental health care declined by 28%, from $3,560 in FY 1995 to $2,562 eight years later, in FY 2003. With inflation adjustment, the reduction in per capita cost would approach 70%. A study of the cost of treating dually diagnosed patients from 1993 to 1997 showed VA costs to be about 10% greater than those of a private sector MarketScan® sample, while costs for patients not dually diagnosed were about 35% greater, although diagnostic severity was far greater in the VA sample (Leslie and Rosenheck, 1999).

Efficiency can be a double-edged sword, whose darker side is represented by reduced treatment resources. While 16% of VA medical center dollars were expended on mental health care in FY 1995, only 11.2% of dollars went to mental health care in FY 2003. Although, in the abstract, increased efficiency cannot be distinguished from reduced service delivery, evidence presented above is generally reassuring since outcomes, where measured, have not deteriorated; there has been no increase in suicide or in VA patients seeking services in other health care systems or becoming incarcerated, and some evidence-based services like the use of atypical antipsychotics and ACT have been expanded in VA. A detailed review of how data were used to guide the transformation of PTSD treatment in VA demonstrates a reasoned approach to system change (Rosenheck and Fontana, 1999). However, the steady decline in the number of veterans who receive specialized SA services from VA, accompanied as it has been by reduced expenditures on SA treatment (Chen et al., 2001), in the presence of epidemiologic data showing no reduction in need (Tessler et al., 2005), has generated expressions of concern among stakeholders (Mulligan, 2003).

Equity Minorities

While there has been widespread documentation of inequities in access and quality of health care to minorities in the US, an extensive series of studies has found little evidence of such inequities in VA, perhaps because, by statute, VA provides cost-free care to all veterans who enroll for services (Greenberg and Rosenheck, 2003; Leda and Rosenheck, 1995; Rosenheck

Suggested Citation:"Appendix C: Mental and Substance Use Health for Veterans: Experience with Performance Evaluation in the Department of Veterans Affairs." Institute of Medicine. 2006. Improving the Quality of Health Care for Mental and Substance-Use Conditions. Washington, DC: The National Academies Press. doi: 10.17226/11470.
×

and Fontana, 1994, 1996a; Rosenheck and Seibyl, 1988; Rosenheck et al., 1997c). However, a 1995 study of over 5,000 veterans treated for PTSD found that on some measures, when African American patients were treated by African American clinicians, they were less likely to drop out of treatment prematurely than when treated by white clinicians (Rosenheck et al., 1995).

Female Veterans

While there has been growing concern that the increasing number of female veterans are reluctant to use VA services, there is evidence that they use VA service at rates equal to or greater than males (Hoff and Rosenheck, 1997, 1998a,b) (in part because of the generally greater inclination of women to use health services). Female veterans express greater or equal levels of satisfaction with VA services (Hoff et al., 1998).

In recognition of the problems of military sexual trauma (and the possible discomfort of women in the virtually all-male VA setting), VA established four Womens Stress Disorder Treatment Teams, which have documented high levels of trauma among women served by these teams (Fontana and Rosenheck, 1997b, 1998). Outcome evaluation found relatively high levels of comfort with the VA setting among traumatized women and no relationship between levels of comfort in VA and outcomes (Fontana and Rosenheck, 2002). Screening for military sexual trauma has recently been introduced as a system-wide performance indicator by OQP.

VI. FRONT-LINE EXPERIENCE

Although there has been no systematic survey of the experiences of front line VA managers and clinicians with the implementation of performance management for MH/SA programs, this account would be incomplete if it did not include at least a few examples of their sometimes ambivalent reactions.

On the one hand, clinicians and their supervisors often report feeling empowered by access to information on both their clinical interventions and on client outcomes. In the past only top medical center management had access to performance data—and the available data were limited in both detail and in quality. In the absence of good data, as Errera observed (Tomich, 1992), decisions tended to reflect the structure of traditional power hierarchies—hierarchies in which MH/SA programs fall toward the bottom. With the increased availability of higher quality information, MH/SA supervisors and clinicians have direct access to information themselves and can more effectively shape and advocate for their programs on the basis of available factual information.

Suggested Citation:"Appendix C: Mental and Substance Use Health for Veterans: Experience with Performance Evaluation in the Department of Veterans Affairs." Institute of Medicine. 2006. Improving the Quality of Health Care for Mental and Substance-Use Conditions. Washington, DC: The National Academies Press. doi: 10.17226/11470.
×

On the other hand, clinicians sometimes complain that gathering data takes time away from patient care and that many measures: (1) are too crude to reflect the complexity of their work, (2) are applied indiscriminantly to highly variable populations (i.e., without risk adjustment for differences in patient characteristics or differences in service environments); and (3) address outcomes over which clinicians feel they have little control, such as treatment dropouts, housing people who are homeless, or even reducing symptoms. Satisfaction surveys, some have complained, in VA as elsewhere, often have quite low response rates and are typically presented without risk adjustment. A recent study of VA performance measurement in JAMA (Walter et al., 2004) presents a set of very sophisticated examples of such problems, elegantly documented by the staff at one VA hospital. While appreciative of the value of performance monitoring, in principle, these investigators identify a number of problems with current practice, largely reflecting application of standards to patients for whom they are clinically inappropriate, (e.g., cancer screening among those who already have terminal illnesses), and it is not hard to imagine that this sophisticated scientific paper had its birth in the discontent of professionals who feel their work has been unfairly judged.

Managers and clinical staff also have observed that many performance measures can be manipulated, and some complain that sites that produce better performance scores have merely been more energetic or inventive in changing various program codes so that patients who do worst are not included in the measure. It is difficult to evaluate these complaints.

Paradoxically, but not surprisingly, it also appears that: (1) the more performance data are used to evaluate high-level managers (and to determine their annual bonuses), (2) the more aggressively middle managers are pressured to get their staffs to improve the scores (often regardless of the availability of new treatments or resources to accomplish these goals), (3) the more likely staff are to feel exploited by their superiors’ quest for bonuses and to look for ways to “game” the system, that is, to improve their numbers without actually changing the care that is given. In many situations, it appears, the limitations of the data are appreciated, and clinicians and managers use them, as intended, to identify ways of improving the care they provide, as best they can. In others, however, it seems there may be more browbeating than joint problem solving.

Reports of falsified or fabricated research data are not unheard of, even in well-funded studies conducted by experienced researchers who have a sophisticated understanding of the importance of adhering to established data collection protocols. It would be naïve to imagine that such activities do not occur among clinical staff who are under pressure to produce the desired results by supervisors who are in a position to influence their future

Suggested Citation:"Appendix C: Mental and Substance Use Health for Veterans: Experience with Performance Evaluation in the Department of Veterans Affairs." Institute of Medicine. 2006. Improving the Quality of Health Care for Mental and Substance-Use Conditions. Washington, DC: The National Academies Press. doi: 10.17226/11470.
×

careers. Use of objective measures, not subject to manipulation, is the best response to this problem.

It must also be noted that even where data or evidence-based practices are well-measured and credible, they do not necessarily determine which way decisions will go. As Paul Errera observed, resource decisions tend to trump performance data. The intensive PTSD program with the best outcomes in the nation was among the first to be closed because it was judged to be inefficient; and monitoring of intensive case management programs like ACT or supported housing shows that the staff/patient ratio (the most resource sensitive measure) is typically the evidence-based performance criterion with the lowest rate of adherence.

There is no way of knowing how widespread these problems are, or to what extent they affect the integrity of performance management in VA MH/SA programs or elsewhere. It seems important to remind ourselves, however, that performance management is ultimately a human process, and that it is affected by the skills, incentives, and integrity of the those whose behavior it seeks to shape.

CONCLUSION

Health care even for a single family or a small clinic can be characterized by both daunting complexity and uncertainty. In a health care system as large as VA, complexity and uncertainty are increased by many orders of magnitude. Every American deserves the best health care possible each time he or she seeks help, and high-quality health care can only be achieved by creative management of the inherent complexity and uncertainty of both illness and health service delivery. Such management demands comprehensive quality monitoring, used by creative and committed leaders, in competent organizations. It is intrinsic to the challenge posed that this work will never be finished, and we probably cannot even know how far along the way we are toward the ideal of improving health outcomes with data. What can be said is that we have made a beginning and we must continue moving forward.

REFERENCES

Ashton CM, Souchek J, Petersen NJ, Menke TJ, Collins TC, Kizer KW, Wright SM, Wray NP. 2003. Hospital use and survival among Veterans Affairs beneficiaries. New England Journal of Medicine 349(17):1637–1646.


Busch S, Leslie D, Rosenheck R. 2004. Measuring quality of pharmacotherapy for depression in a national health care system. Medical Care 42(6):532–542.


Castells M. 2000. The Rise of the Network Society. 2nd ed. Malden, MA: Blackwell.

Chen S, Wagner TH, Barnett PG. 2001. The effect of reforms on spending for veterans’ substance abuse treatment, 1993–1999. Health Affairs 20(4):169–175.

Suggested Citation:"Appendix C: Mental and Substance Use Health for Veterans: Experience with Performance Evaluation in the Department of Veterans Affairs." Institute of Medicine. 2006. Improving the Quality of Health Care for Mental and Substance-Use Conditions. Washington, DC: The National Academies Press. doi: 10.17226/11470.
×

Desai MM, Rosenheck RA, Druss BG, Perlin JB. 2002a. Mental disorders and quality of diabetes care. American Journal of Psychiatry 159(9):1584–1590.

Desai MM, Rosenheck RA, Druss BG, Perlin JB. 2002b. Mental disorders and quality of care among post-acute myocardial infarction outpatients. Journal of Nervous and Mental Disease 190(1):51–53.

Desai MM, Rosenheck RA, Druss BG, Perlin JB. 2002c. Receipt of nutrition and exercise counseling among medical outpatients with psychiatric and substance abuse disorders. Journal of General Internal Medicine 17(7):556–560.

Desai MM, Rosenheck RA, Kasprow W. 2003. Determinants of receipt of medical care in a national sample of homeless veterans. Medical Care 41(2):275–287.

Desai RA, Rosenheck RA. 2000. The interdependence of mental health service systems: The effects of VA mental health funding on veterans’ use of state mental health inpatient facilities. Journal of Mental Health Policy and Economics 3(2):61–68.

Desai R, Rosenheck RA. 2002. Service use among VA mental health patients in Colorado: The impact of managed care on cross-system service use. Psychiatric Services 53(12): 1599–1605.

Desai RA, Rosenheck RA, Rothbard A. 2001. Cross system service use among VA mental health patients in Philadelphia. Administration and Policy in Mental Health 28(4): 299–309.

Desai RA, Dausey D, Rosenheck RA. 2005. Mental health service delivery and suicide risk: The role of individual patient and facility factors. American Journal of Psychiatry 162(2):311–318.

Desai RA, Rosenheck RA, Sernyak MJ, Dausey D. (In press-a). A comparison of service delivery by the Department of Veterans Affairs and State providers: The Role of Academic Affiliation. Administration and Policy in Mental Health.

Desai RA, Dausey D, Rosenheck RA. (In press-b). Suicide in a national sample of psychiatric patients: The role of gun ownership, legislation, and social capital.

Druss BG, Rosenheck RA. 1999. Patient satisfaction and administrative measures as indicators of the quality of mental health care. Psychiatric Services 50(8):1053–1058.

Druss BG, Rohrbaugh RM, Levinson CM, Rosenheck RA. 2001. Integrated medical care for patients with serious psychiatric illness: A randomized trial. Archives of General Psychiatry 58(9):861–868.

Druss BG, Rosenheck RA, Desai MM, Perlin JB. 2002. Quality of preventive medical care for patients with mental disorders. Medical Care 40(2):129–136.


Errera P. 1988. From Yale professor to Washington bureaucrat: Policy and medicine. VA Practitioner 81–93.


Fontana AF, Rosenheck RA. 1997a. Effectiveness and cost of inpatient treatment of posttraumatic stress disorder. American Journal of Psychiatry 154(6):758–765.

Fontana AF, Rosenheck RA. 1997b. Women under Stress: Evaluation of the Women’s Stress Disorder Treatment Teams. West Haven, CT: Northeast Program Evaluation Center.

Fontana AF, Rosenheck RA. 1998. Focus on women: Duty-related and sexual stress in the etiology of PTSD among women veterans who seek treatment. Psychiatric Services 49(5):658–662.

Fontana AF, Rosenheck RA. 2001. A model of patient’s satisfaction with treatment for posttraumatic stress disorder. Administration and Policy in Mental Health 28(6):475–489.

Fontana A, Rosenheck RA. 2002. Women under Stress II: Evaluation of the Clinical Performance of the Department of Veterans Affairs Women’s Stress Disorder Treatment Teams. West Haven, CT: Northeast Program Evaluation Center.

Fontana AF, Ford JD, Rosenheck RA. 2003. A multivariate model of patients’ satisfaction with treatment for posttraumatic stress disorder. Journal of Traumatic Stress Studies 16(1):93–106.

Suggested Citation:"Appendix C: Mental and Substance Use Health for Veterans: Experience with Performance Evaluation in the Department of Veterans Affairs." Institute of Medicine. 2006. Improving the Quality of Health Care for Mental and Substance-Use Conditions. Washington, DC: The National Academies Press. doi: 10.17226/11470.
×

Gamache G, Rosenheck RA, Tessler R. 2000. Choice of provider among homeless people with mental illness: Veterans and the VA. Psychiatric Services 51(8):1024–1017.

Gamache G, Rosenheck RA, Tessler R. 2001. The proportion of veterans among homeless men: A decade later. Social Psychiatry and Psychiatric Epidemiology 36(10):481–485.

Gamache G, Rosenheck RA, Tessler R. 2003. Overrepresentation of women veterans among homeless women. American Journal of Public Health 93(7):1132–1136.

GAO (General Accounting Office). 2001. VA Health Care: Community-Based Clinics Improve Primary Care Access. GAO-01-678T. Washington, DC.

Goldman HH, Morrissey J, Rosenheck RA, Cocozza J, Randolph F, Blasinsky M, ACCESS National Evaluation Team. 2002. Lessons from the evaluation of the ACCESS program. Psychiatric Services 53(8):967–970.

Greenberg GA, Rosenheck R. 2003. Change in mental health service delivery among blacks, whites and Hispanics in the Department of Veterans Affairs. Administration and Policy in Mental Health 31(1):31–45.

Greenberg GA, Rosenheck RA. 2004a. National Mental Health Program Performance Monitoring System: Fiscal Year 2003 Report, West Haven, CT: Northeast Program Evaluation Center.

Greenberg GA, Rosenheck RA. 2004b. Changes in satisfaction with mental health services among blacks, whites and Hispanics in the Department of Veterans Affairs. Psychiatric Quarterly 75(4):375–389.

Greenberg GA, Rosenheck RA, Seibyl CL. 2002. Continuity of care and clinical effectiveness: Outcomes following residential treatment for severe substance abuse. Medical Care 40(3):246–259.

Greenberg GA, Rosenheck RA, Fontana A. 2003. Continuity of care and clinical effectiveness: Treatment of posttraumatic stress disorder in the Department of Veterans Affairs. Journal of Behavioral Health Services and Research 30(2):202–214.


HEDIS (Health Employer Data and Information Set). 1999. 2000: Health Employer Data and Information Set. Washington, DC: National Committee on Quality Improvement.

Hoff RA, Rosenheck RA. 1997. Utilization of mental health services by women in a male-dominated environment: The VA experience. Psychiatric Services 48(11):1408–1414.

Hoff RA, Rosenheck RA. 1998a. Female veterans use of VA health care services. Medical Care 36(7):1114–1119.

Hoff RA, Rosenheck RA. 1998b. The use of VA and non-VA mental health services by female veterans. Medical Care 36(11):1524–1533.

Hoff RA, Rosenheck RA, Wilson N, Meterko M. 1998. Quality of VA mental health service delivery. Administration and Policy in Mental Health 26(1):214–218.


IOM (Institute of Medicine). 2001. Crossing the Quality Chasm. Washington, DC: National Academy Press.


Jha AK, Perlin JB, Kizer KW, Dudly RA. 2003. Effect of the transformation of Veterans Affairs health care system on the quality of care. New England Journal of Medicine 348(22):2218–2227.


Kasprow WJ, Rosenheck RA, Frisman LK. 1999. Homeless veterans’ satisfaction with residential treatment. Psychiatric Services 50(4):540–546.

Kasprow WJ, Rosenheck RA, Dilella D, and Cavallaro L. 2002. Health Care for Homeless Veterans programs: Fifteenth Progress Report. West Haven, CT: Northeast Program Evaluation Center.

Kasprow WJ, Rosenheck RA, Dilella D, Cavallaro L, Harelik N. 2004. Health Care for Homeless Veterans Programs: Seventeenth Progress Report. West Haven, CT: Northeast Program Evaluation Center [Report to Congress].

Kizer KW. 1995. Vision for Change: A Plan to Restructure the Veterans Health Administration. Washington, DC: Department of Veterans Affairs.

Suggested Citation:"Appendix C: Mental and Substance Use Health for Veterans: Experience with Performance Evaluation in the Department of Veterans Affairs." Institute of Medicine. 2006. Improving the Quality of Health Care for Mental and Substance-Use Conditions. Washington, DC: The National Academies Press. doi: 10.17226/11470.
×

Kizer KW. 1996. Prescription for Change: The Guiding Principals and Strategic Objectives Underlying the Transformation of the Veterans Health Administration. Washington, DC: Department of Veterans Affairs.

Kizer KW. 1999. The “New VA”: A national laboratory for health care quality management. American Journal of Medical Quality 14(1):3–20.

Kizer KW, Garthwaite TG. 1997. Vision for change: An integrated service network. In: Kolodner RM, ed. Computerizing Large Integrated Health Networks: The VA Success. New York, NY: Springer-Verlag. Pp. 3–13.

Krystal JH, Cramer JA, Krol WF, Kirk GF, Rosenheck RA, and the Veterans Affairs Naltrexone Cooperative Study 425 Group. 2001. Naltrexone in the treatment of alcohol dependence. New England Journal of Medicine 345(24):1734–1739.

Kulka R, Schlenger W, Fairbanks J, Hough R, Jordan B, Marmar C, Weiss D. 1990. Trauma and the Vietnam War Generation: Report of Findings from the National Vietnam Veterans Readjustment Study. New York: Brunner/Mazel.


Leda C, Rosenheck RA. 1995. Race in the treatment of homeless mentally ill veterans. Journal of Nervous and Mental Disease 183(8):529–537.

Leslie DL, Rosenheck RA. 1999. Inpatient treatment of comorbid psychiatric and substance abuse disorders: A comparison of public sector and privately insured populations. Administration and Policy in Mental Health 26(4):253–268.

Leslie DL, Rosenheck R. 2000. Comparing quality of mental health care in public sector and privately insured populations: First efforts and methodological challenges. Psychiatric Services 51(5):650–655.

Leslie DL, Rosenheck RA. 2001. The effect of institutional fiscal stress on the use of atypical antipsychotic medications in the treatment of schizophrenia. Journal of Nervous and Mental Disease 189(6):377–383.

Leslie DL, Rosenheck RA. 2003a. Benchmarking the quality of schizophrenia pharmacotherapy: A comparison of the Department of Veterans Affairs and the private sector. Journal of Mental Health Policy and Economics 6(3):113–121.

Leslie DL, Rosenheck RA. 2003b. Fourth Annual Report on Pharmacotherapy of Schizophrenia in the Department of Veterans Affairs. West Haven, CT: Northeast Program Evaluation Center.

Leslie DL, Rosenheck RA. 2005. Pharmacotherapy and healthcare costs among patients with schizophrenia and newly diagnosed diabetes. Psychiatric Services 56(7):803–810.

Linn MW, Caffey EM, Klett CJ, Hogarty G. 1977. Hospital vs. community (foster) care for psychiatric patients. Archives of General Psychiatry 34(1):78–83.

Linn MW, Caffey EM, Klett CJ, Hogarty G, Lamb HR. 1979. Day treatment and psychotropic drugs in the aftercare of schizophrenic patients. Archives of General Psychiatry 36(10):1055–1066.


McGuire J, Rosenheck R, Burnette C. 2002. Expanding service delivery: Does it improve relationships among agencies serving homeless people with mental illness? Administration and Policy in Mental Health 29(3):243–256.

McGuire J, Rosenheck RA, Kasprow W. 2003. Health status, service use, and costs among veterans receiving outreach services in jails or community settings. Psychiatric Services 54(2):201–207.

Meyer J, Rowan B. 1977. Institutionalized organizations: Formal structure as myth and ceremony. American Journal of Sociology 83:440–463.

Mojtabi R, Rosenheck R, Wyatt RJ, Susser E. 2003. Transition to VA outpatient mental health services among severely mentally ill patients discharged from the armed services. Psychiatric Services 54(3):383–388.

Suggested Citation:"Appendix C: Mental and Substance Use Health for Veterans: Experience with Performance Evaluation in the Department of Veterans Affairs." Institute of Medicine. 2006. Improving the Quality of Health Care for Mental and Substance-Use Conditions. Washington, DC: The National Academies Press. doi: 10.17226/11470.
×

Moos RH, Finney JW, Federman EB, Suchinsky R. 2000. Specialty mental health care improves patients’ outcomes: Findings from a nationwide program to monitor the quality of care for patients with substance abuse disorders. Journal of Studies on Alcohol 61:704–713.

Mulligan K. 2002. VA allowing mental health care to erode APA charges. Psychiatric News 37(7):2.

Mulligan K. 2003. VA Accused of shortchanging substance abuse treatment. Psychiatric News 38(9):4.


Neale M, Rosenheck R, Martin A, Morrissey J, Castrodonatti J. 2003. Mental Health Intensive Case Management (MHICM): The Sixth National Performance Monitoring Report: FY 2002. West Haven, CT: Northeast Program Evaluation Center.

Norquist GS, Hough RL, Golding JM, Escobar JI. 1990. Psychiatric disorder in male veterans and nonveterans. Journal of Nervous and Mental Disease 178(5):328–335.


Pandiani JA, Banks SM, Schacht LM. 1998. Personal privacy versus public accountability: A technological solution to an ethical dilemma. Journal of Behavioral Health Services and Research 25(4):456–463.

Peterson PE. 1995. The Price of Federalism. Washington, DC: Brookings Institution.

Petrakis I, Leslie D, Rosenheck RA. 2003a. The use of antidepressants in alcohol dependent veterans. Journal of Clinical Psychiatry 64(8):865–870.

Petrakis I, Leslie D, Rosenheck RA. 2003b. Use of Naltrexone in the Treatment of Alcoholism Nationally in the Department of Veterans Affairs. Alcoholism: Clinical and Experimental Research 27(11):1780–1784.

President’s New Freedom Commission on Mental Health. 2003. Achieving the Promise: Transforming Mental Health Care in America. Subcommittee on Acute Care Report. DHHS Pub. No. SAM-03-3832. Rockville, MD: U.S. Department of Health and Human Services.

Pressman J, Wildavsky A. 1971. Implementation: How Great Expectations in Washington are Dashed in Oakland. Berkeley, CA: University of California Press.


Resnick S, Rosenheck RA, Lehman A. 2004a. An exploratory analysis of correlates of recovery. Psychiatric Services 55(5):540–547.

Resnick S, Armstrong M, Sperazza M, Harkness L, Rosenheck RA. 2004b. A model of consumer provider partnership: Vet-to-vet. Psychiatric Rehabilitation Journal 28(2): 185–187.

Rosenheck RA. 2001a. Organizational process: A missing link between research and practice. Psychiatric Services 52(12):1607–1612.

Rosenheck RA. 2001b. Stages in the implementation of innovative clinical programs in complex organizations. Journal of Nervous and Mental Disease 189(12):812–821.

Rosenheck RA, Cicchetti D. 1995. A Mental Health Program Performance Monitoring System for the Department of Veterans Affairs. West Haven, CT: Northeast Program Evaluation Center.

Rosenheck RA, Cicchetti D. 1998. A mental health program report card: A multidimensional approach to performance monitoring in public sector programs. Community Mental Health Journal 34(1):85–106.

Rosenheck RA, Fontana AF. 1994. Utilization of mental health services by minority veterans of the Vietnam era. Journal of Nervous and Mental Disease 182:685–691.

Rosenheck RA, Fontana AF. 1995. Do Vietnam era veterans who suffer from posttraumatic stress disorder avoid VA mental health services? Military Medicine 160:136–142.

Rosenheck RA, Fontana AF. 1996a. Race and outcome of treatment for veterans suffering from PTSD. Journal of Traumatic Stress Studies 9(2):343–351.

Suggested Citation:"Appendix C: Mental and Substance Use Health for Veterans: Experience with Performance Evaluation in the Department of Veterans Affairs." Institute of Medicine. 2006. Improving the Quality of Health Care for Mental and Substance-Use Conditions. Washington, DC: The National Academies Press. doi: 10.17226/11470.
×

Rosenheck RA, Fontana AF. 1996b. Treatment of veterans severely impaired by PTSD. In Ursano RJ, Norwood AE, eds. Emotional Aftermath of the Persian Gulf War. Washington, DC: American Psychiatric Press.

Rosenheck RA, Fontana A. 1999. Changing patterns of care for war-related post-traumatic stress disorder at Department of Veterans Affairs medical centers: The use of performance data to guide program development. Military Medicine 164(11):795–802.

Rosenheck RA, Fontana A. 2001. Impact of efforts to reduce inpatient costs on clinical effectiveness: Treatment of post traumatic stress disorder in the Department of Veterans Affairs. Medical Care 39(2):168–180.

Rosenheck RA, Fontana AF. 2002. African-American and Latino veterans in intensive VA treatment programs for posttraumatic stress disorder. Medical Care 40(Supplement 1): I52–I61.

Rosenheck RA, Koegel P. 1993. Characteristics of veterans and nonveterans in three samples of homeless men. Hospital and Community Psychiatry 44:858–863.

Rosenheck RA, Massari LA. 1993. Wartime military service and utilization of VA health care services. Military Medicine 158:223–228.

Rosenheck RA, Neale MS. 1998. Cost-effectiveness of intensive psychiatric community care for high users of inpatient services. Archives of General Psychiatry 55(5):459–466.

Rosenheck RA, Seibyl CL. 1998. The experience of black and white veterans in a residential treatment and work therapy program for substance abuse. American Journal of Psychiatry 155(8):1029–1034.

Rosenheck RA, Stolar M. 1998. Access to public mental health services: Determinants of population coverage. Medical Care 36(4):503–512.

Rosenheck RA, Fontana AF, Cottrel C. 1995. Effect of clinician-veteran racial pairing in the treatment of posttraumatic stress disorder. American Journal of Psychiatry 152(4): 555–563.

Rosenheck RA, Leda C, Frisman LK, Lam J, Chung A. 1996a. Homeless veterans. In: Baumohl J, ed. Homelessness in America: A Reference Book. Phoenix, AZ: Oryx Press.

Rosenheck RA, Leda C, Sieffert D, Burnette C. 1996b. Fiscal Year 1995 End-of-Year Survey of Homeless Veterans in VA Inpatient and Domiciliary Care Programs. West Haven, CT: Northeast Program Evaluation Center.

Rosenheck RA, Cramer J, Xu W, Thomas J, Henderson W, Frisman LK, Fye C, Charney D. 1997a. A comparison of clozapine and haloperidol in the treatment of hospitalized patients with refractory schizophrenia. New England Journal of Medicine 337(12): 809–815.

Rosenheck RA, Wilson N, Meterko M. 1997b. Consumer satisfaction with inpatient mental health treatment: Influence of patient and hospital factors. Psychiatric Services 48:1553–1561.

Rosenheck RA, Leda C, Frisman LK, Gallup P. 1997c. Homeless mentally ill veterans: Race, service use and treatment outcome. American Journal of Orthopsychiatry 67(4): 632–639.

Rosenheck RA, Fontana A, Stolar M. 1999a. Assessing Quality of Care: Administrative Indicators and Clinical Outcomes in Posttraumatic Stress Disorder. Medical Care. 37(2): 180–188.

Rosenheck RA, Frisman LK, Kasprow W. 1999b. Improving access to disability benefits among homeless persons with mental illness: An agency-specific approach to services integration. American Journal of Public Health 89(4):524–528.

Rosenheck RA, Hoff RA, Steinwachs D, Lehman A. 2000a. Benchmarking treatment of schizophrenia: A comparison of service delivery by the national government and by state and local providers. Journal of Nervous and Mental Disease 188(4):209–216.

Suggested Citation:"Appendix C: Mental and Substance Use Health for Veterans: Experience with Performance Evaluation in the Department of Veterans Affairs." Institute of Medicine. 2006. Improving the Quality of Health Care for Mental and Substance-Use Conditions. Washington, DC: The National Academies Press. doi: 10.17226/11470.
×

Rosenheck RA, Dausey D, Frisman LK, Kasprow W. 2000b. Impact of receipt of social security benefits on homeless veterans with mental illness. Psychiatric Services 51(12):1 549–1554.

Rosenheck RA, Banks S, Pandiani J. 2000c. Does closing beds in one public mental health system result in increased use of hospital services in other systems? Mental Health Services Research 2(4):183–189.

Rosenheck RA, Banks S, Pandiani J, Hoff R. 2000d. Bed closures and incarceration rates among users of Veterans Affairs mental health services. Psychiatric Services 51(10):1282–1287.

Rosenheck RA, Leslie DL, Sernyak M. 2001a. From clinical trials to real-world practice: Use of atypical antipsychotic medication nationally in the Department of Veterans Affairs. Medical Care 39(3):302–308.

Rosenheck RA, Frisman LK, Essock S. 2001b. Impact of VA bed closures on use of state mental health services. Journal of Behavioral Health Services and Research 28(1):58–66.

Rosenheck RA, Lam J, Morrissey JP, Calloway M, Stolar M, Randolph F, and the ACCESS National Evaluation Team. 2002. Service systems integration and outcomes for mentally ill homeless persons in the ACCESS program. Psychiatric Services 53(8):958–966.

Rosenheck RA, Kasprow W, Frisman LK, Liu-Mares W. 2003a. Cost-effectiveness of supported housing for homeless persons with mental illness. Archives of General Psychiatry 60(9):940–951.

Rosenheck RA, Desai R, Kasprow, Mares A. 2003b. Progress report on new initiatives for Homeless Veterans from the Veterans Health Administration.West Haven, CT: Northeast Program Evaluation Center.

Rosenheck RA, Leslie D, Woody G. 2003c. Fiscal strain and access to opiate substitution therapy at Department of Veterans Affairs Medical Centers. American Journal on Addictions 12(3):220–228.

Rosenheck R, Perlick D, Bingham S, Liu-Mares W, Collins J, Warren S, Leslie D, Allan E, Campbell C, Caroff S, Corwin J, Davis L, Douyon R, Dunn L, Evans D, Frecska E, Grabowski J, Graeber D, Herz L, Kwon K, Lawson W, Mena F, Sheikh J, Smelson D, Smith-Gamble V. 2003d. Effectiveness and cost of olanzapine and haloperidol in the treatment of schizophrenia: a randomized controlled trial. Journal of the American Medical Association 290(20):2693–2702.


Seibyl CL, Rosenheck RA, Siefert D, Medak S. 2001. Fiscal Year 2000 End-of-Year Survey of Homeless Veterans in VA Inpatient Programs. West Haven, CT: Northeast Program Evaluation Center.

Seibyl CL, Rosenheck RA, Corwel L, Medak S. 2003. Sixth Progress Report on the Compensated Work Therapy/Veterans Industries Program. Fiscal Year 2002. West Haven, CT: Northeast Program Evaluation Center.

Sernyak MJ, Rosenheck RA. 2003. Risk adjustment in studies using administrative data. Schizophrenia Bulletin 29(2):267–271.

Sernyak MJ, Leslie D, Alarcon R, Losonczy M, Rosenheck RA. 2002. Association of diabetes mellitus with use of atypical neuroleptics in the treatment of schizophrenia. American Journal of Psychiatry 159(4):561–566.

Sernyak MJ, Gulanski B, Leslie DL, Rosenheck RA. 2003. Undiagnosed hyperglycemia in clozapine-treated patients with schizophrenia. Journal of Clinical Psychiatry 64(5): 605–608.

Smith SR, Lipsky M. 1993. Nonprofits for Hire: The Welfare State in the Age of Contracting. Cambridge, MA: Harvard University Press.

Stein LI, Test MA. 1980. Alternative to mental hospital treatment. I. Conceptual model, treatment program and clinical evaluation. Archives of General Psychiatry 37(4): 392–397.

Suggested Citation:"Appendix C: Mental and Substance Use Health for Veterans: Experience with Performance Evaluation in the Department of Veterans Affairs." Institute of Medicine. 2006. Improving the Quality of Health Care for Mental and Substance-Use Conditions. Washington, DC: The National Academies Press. doi: 10.17226/11470.
×

Tessler R, Rosenheck RA, Gamache G. 2002. Comparison of homeless veterans with other homeless men in a large clinical outreach program. Psychiatric Quarterly 73(2): 109–119.

Tessler R, Rosenheck RA, Gamache G. 2005. Declining access to alcohol and drug abuse services among veterans in the general population. Military Medicine 170(3):234–238.

Tomich N. 1992. Paul Errera: “One does have choices.” US Medicine 28:1.


U.S. Census Bureau. 2001. Statistical Abstract of the United States. 121st ed. Washington, DC: U.S. Census Bureau.

U.S. Department of Veterans Affairs. 2003. FY 2003 Annual performance and Accountability Report. Washington, DC: U.S. Department of Veterans Affairs.

USVA (U.S. Veterans Affairs). 2004. [Online]: http://www.va.gov/vetdata/SurveyResults [December 5, 2005].


Walter L, Davidowitz N, Heineken P. Kovinsky K. 2004. Pitfalls of converting practice guidelines into quality measures: Lessons learned from a VA performance measure. Journal of the American Medical Association 291:2466–2470.

Weissman EM, Rosenheck RA, Essock SM. 2002. Impact of modifying risk adjustment models on rankings of access to care in the VA mental health report card. Psychiatric Services 53(9):1153–1158.

Suggested Citation:"Appendix C: Mental and Substance Use Health for Veterans: Experience with Performance Evaluation in the Department of Veterans Affairs." Institute of Medicine. 2006. Improving the Quality of Health Care for Mental and Substance-Use Conditions. Washington, DC: The National Academies Press. doi: 10.17226/11470.
×

TABLE 1 Population Characteristics and Percentage of Veterans Who Used VA Services, 2001

 

(1) Total Population Characteristics

(2) Percent Who Used VA Health Services: Past Year

(3) Percent Who Used VA Health Services: Lifetime

(4) Characteristics of Veterans Who Used MH/SA Services: Past Year

(5) % of MH/SA Services Users Who Used VHA MH/SA Services

[Population] Size

25,196,036

5,161,036

8,675,570

1,676,170

681,969

Proportion of All Veterans

 

20.5%

34.4%

6.6%

2.7%

Who Used Services

 

 

 

 

(41.0% of Col 4)

Sociodemographic

Average Age

57.7

61.7

59.8

50.5

53.4

Age

<30

3.8%

12.3%

21.2%

5.3%

22.9%

30–49

25.6%

15.0%

31.3%

41.6%

32.3%

50–59

24.6%

18.4%

35.1%

32.8%

48.6%

60–75

29.6%

23.4%

34.0%

14.2%

46.1%

>75

16.4%

29.0%

42.6%

6.2%

52.0%

Race*

White

83.3%

19.1%

32.4%

76.8%

37.1%

Black

8.8%

30.7%

48.7%

12.2%

51.8%

Hispanic

4.1%

20.1%

35.5%

4.7%

47.1%

Other

3.9%

27.8%

44.4%

6.3%

54.1%

Gender

Male

94.1%

20.6%

34.6%

87.7%

42.2%

Female

5.9%

17.9%

32.5%

12.3%

28.4%

Education

<High School Graduate

11.2%

33.9%

46.0%

8.37%

54.0%

High School Graduate

30.0%

21.4%

33.3%

27.8%

45.5%

Post High School Education

35.8%

19.8%

35.0%

42.4%

39.6%

4 Yr College Degree or Higher

23.0%

13.9%

29.3%

21.5%

30.2%

Suggested Citation:"Appendix C: Mental and Substance Use Health for Veterans: Experience with Performance Evaluation in the Department of Veterans Affairs." Institute of Medicine. 2006. Improving the Quality of Health Care for Mental and Substance-Use Conditions. Washington, DC: The National Academies Press. doi: 10.17226/11470.
×

Employment

Employed

55.1%

12.8%

28.2%

46.8%

25.6%

Retired/Disabled

39.1%

30.3%

42.1%

39.3%

57.7%

Unemployed/Other

5.9%

26.8%

41.2%

13.8%

42.2%

Marital Status

Married

75.1%

18.6%

32.3%

59.1%

36.2%

Widowed

5.31%

25.5%

40.3%

3.0%

33.2%

Separated/Divorced

12.43%

29.1%

45.5%

25.8%

49.0%

Never Married

7.18%

22.0%

33.6%

12.1%

45.1%

Income

<$20,000

17.0%

43.8%

56.4%

27.9%

59.6%

$20,000–$40,000

27.9%

25.6%

38.6%

30.5%

48.7%

>$40,000

55.2%

10.2%

25.4%

41.7%

20.7%

Insurance

Medicaid or Medicare

40.1%

28.7%

41.2%

31.5%

51.0%

Private

63.3%

10.8%

26.0%

49.3%

20.7%

Medigap

18.2%

26.1%

38.7%

6.9%

40.1%

Military Related (for example, CHAMPUS)

7.2%

27.6%

45.7%

8.2%

49.7%

Other Government Insurance (for example, Indian Health Service)

3.5%

27.7%

45.3%

4.0%

58.5%

Military Experience

Active Duty

0–2 years

33.2%

20.0%

33.5%

32.1%

39.8%

3–5 years

46.1%

19.1%

33.0%

42.8%

38.2%

5 years or more

20.7%

24.5%

39.5%

25.1%

46.1%

Served in a Combat or War Zone

39.3%

26.2%

42.5%

44.9%

50.4%

Exposed to Dead or Wounded

36.7%

26.0%

42.6%

49.8%

48.9%

Suggested Citation:"Appendix C: Mental and Substance Use Health for Veterans: Experience with Performance Evaluation in the Department of Veterans Affairs." Institute of Medicine. 2006. Improving the Quality of Health Care for Mental and Substance-Use Conditions. Washington, DC: The National Academies Press. doi: 10.17226/11470.
×

 

(1) Total Population Characteristics

(2) Percent Who Used VA Health Services: Past Year

(3) Percent Who Used VA Health Services: Lifetime

(4) Characteristics of Veterans Who Used MH/SA Services: Past Year

(5) % of MH/SA Services Users Who Used VHA MH/SA Services

Prisoner of War

.5%

49.3%

64.4%

1.2%

71.6%

Exposed to Environmental Hazards

19.3%

27.3%

45.0%

38.9%

51.1%

Period of Service**

WWI through WWII

20.5%

28.2%

41.5%

7.5%

50.0%

Btwn WWII & Korean War

2.4%

28.8%

39.9%

1.0%

68.6%

Korean War

12.2%

25.7%

37.5%

5.5%

54.0%

Btwn Korean & Vietnam War

16.3%

19.3%

29.1%

11.8%

46.6%

Vietnam Era

25.6%

18.0%

37.3%

40.3%

45.1%

Post-Vietnam Era

17.7%

13.9%

27.8%

25.2%

29.6%

Persian Gulf War Era (1990–)

4.9%

12.1%

22.3%

8.2%

24.2%

Health Status

Self-Reported Health Status

Fair-Poor

24.2%

39.9%

54.0%

45.2%

55.2%

Good

30.2%

19.9%

35.4%

27.3%

38.2%

Excellent-Very Good

45.6%

9.8%

22.7%

27.5%

17.4%

Any Health Service Use For

Alcohol or Drugs

1.2%

43.1%

59.4%

10.9%

41.6%

PTSD

3.8%

57.3%

73.0%

32.5%

60.3%

Mental Health

6.2%

43.2%

58.1%

52.8%

44.4%

Any

8.9%

45.1%

61.1%

68.9%

46.8%

Suggested Citation:"Appendix C: Mental and Substance Use Health for Veterans: Experience with Performance Evaluation in the Department of Veterans Affairs." Institute of Medicine. 2006. Improving the Quality of Health Care for Mental and Substance-Use Conditions. Washington, DC: The National Academies Press. doi: 10.17226/11470.
×

Service Connected

Not Service Connected

87.8%

15.9%

28.5%

69.6%

26.1%

<50%

9.6%

43.9%

70.5%

17.3%

63.3%

>50%

2.6%

80.3%

91.2%

13.0%

84.0%

Mental health interferes with work or activities at all

49.9%

29.9%

45.0%

83.0%

43.3%

*Exclusive categories (veterans in more than one category were classified as other).

**Exclusive categories (first period of service used).

Suggested Citation:"Appendix C: Mental and Substance Use Health for Veterans: Experience with Performance Evaluation in the Department of Veterans Affairs." Institute of Medicine. 2006. Improving the Quality of Health Care for Mental and Substance-Use Conditions. Washington, DC: The National Academies Press. doi: 10.17226/11470.
×

TABLE 2 Logistic Regression Analysis of VA Health Service Use: Factors That Predict Recent and Lifetime Use

 

Used Service During the Past Year

Ever Used Services

Odds Ratio

95% Confidence Interval

Rank Order

Odds Ratio

95% Confidence Interval

Rank Order

Sociodemographic

Age

<30

0.58*

0.57-0.58

8

0.63**

0.63-0.64

10

30–49

0.69

0.68-0.69

16

1.11

1.10-1.11

28

50–59

1.01

1.00-1.01

38

1.32

1.31-1.33

17

60–75

1.15

1.14-1.16

24

1.05

1.04-1.05

31

>75

1.00

Reference

1.00

Reference

Race

Black

1.57

1.56-1.58

13

1.74

1.73-1.74

7

Hispanic

0.98

0.97-0.99

36

1.13

1.13-1.14

26

Other

1.33

1.32-1.34

19

1.35

1.35-1.36

15

Whites

1.00

Reference

1.00

Reference

Gender

Male

1.00

Reference

1.00

Reference

Female

1.08

1.07-1.08

29

1.08

1.07-1.08

30

Education

<High School Graduate

1.00

Reference

1.00

Reference

High School Graduate

0.90

0.90-0.90

28

0.89

0.89-0.90

27

Post High School Education

0.94

0.93-0.94

31

1.04

1.04-1.04

32

4 Yr College Degree or Higher

0.78

0.78-0.78

20

1.00

1.00-1.00

38

Employment

Employed

1.00

Reference

1.00

Reference

Retired/Disabled

1.11

1.11-1.12

27

0.99

0.98-0.99

36

Unemployed/Other

1.07

1.06-1.07

33

0.96

0.96-0.97

33

Suggested Citation:"Appendix C: Mental and Substance Use Health for Veterans: Experience with Performance Evaluation in the Department of Veterans Affairs." Institute of Medicine. 2006. Improving the Quality of Health Care for Mental and Substance-Use Conditions. Washington, DC: The National Academies Press. doi: 10.17226/11470.
×

Marital Status

Married

1.00

Reference

1.00

Reference

Widowed

0.68

0.68-0.69

15

0.85

0.84-0.85

23

Separated/Divorced

1.07

1.07-1.07

32

1.20

1.20-1.20

21

Never Married

0.96

0.95-0.96

34

0.84

0.84-0.85

22

Income

<$20,000

1.00

Reference

1.00

Reference

$20,000–$40,000

0.63

0.63-0.63

10

0.61

0.61-0.61

9

>$40,000

0.34

0.34-0.34

3

0.44

0.44-0.44

3

Income Information Missing

0.57

0.57-0.57

7

0.54

0.54-0.54

5

Insurance

Medicaid or Medicare

1.01

1.01-1.01

37

1.17

1.16-1.17

24

Private

0.35

0.35-0.35

4

0.57

0.57-0.57

6

Medigap

0.90

0.90-0.90

26

0.97

0.96-0.97

35

Military Related (for example, CHAMPUS)

0.85

0.85-0.86

21

0.96

0.96-0.97

34

Other Government Insurance (for example, Indian Health Service)

1.16

1.16-1.17

23

1.26

1.26-1.27

19

Military Experience

Active Duty

0–2 years

1.00

Reference

1.00

Reference

3–5 years

1.04

1.04-1.04

35

1.00

1.00-1.01

37

5 years or more

1.16

1.16-1.17

22

0.92

0.92-0.92

29

Served in a Combat or War Zone

1.15

1.14-1.15

25

1.26

1.25-1.26

20

Exposed to Dead or Wounded

1.07

1.07-1.07

30

1.15

1.15-1.16

25

Prisoner of War

1.60

1.58-1.63

9

1.52

1.50-1.55

12

Suggested Citation:"Appendix C: Mental and Substance Use Health for Veterans: Experience with Performance Evaluation in the Department of Veterans Affairs." Institute of Medicine. 2006. Improving the Quality of Health Care for Mental and Substance-Use Conditions. Washington, DC: The National Academies Press. doi: 10.17226/11470.
×

 

Used Service During the Past Year

Ever Used Services

Odds Ratio

95% Confidence Interval

Rank Order

Odds Ratio

95% Confidence Interval

Rank Order

Health Status

Self-Reported Health Status

Fair-Poor

1.91

1.92-1.95

5

1.71

1.71-1.72

8

Good

1.40

1.39-1.40

18

1.32

1.32-1.32

18

Excellent-Very Good

1.00

Reference

1.00

Reference

Any Health Service Use For

Alcohol or Drugs

1.43

1.41-1.44

17

1.40

1.38-1.41

14

PTSD

1.58

1.57-1.59

12

1.58

1.57-1.59

11

Mental Health

1.58

1.58-1.59

11

1.40

1.40-1.41

13

Service Connected

Not Service Connected

1.00

Reference

1.00

Reference

<50%

3.66

3.65-3.68

2

5.21

5.19-5.23

2

>50%

12.10

12.00-12.19

1

14.12

13.98-14.26

1

Mental health interferes with work or activities at all

1.47

1.47-1.48

14

1.33

1.33-1.33

16

Proxy Answered Questions

1.85

1.84-1.87

6

1.91

1.90-1.92

4

*All odds ratios for whether used service in the past year were significant at p<.0001 except for age 50–59, which was not sign ificant.

**All odds ratios for whether ever used services were significant at p<.0001, except for having a college education or higher and years of service from 3 to 5 years, which were not significant.

Suggested Citation:"Appendix C: Mental and Substance Use Health for Veterans: Experience with Performance Evaluation in the Department of Veterans Affairs." Institute of Medicine. 2006. Improving the Quality of Health Care for Mental and Substance-Use Conditions. Washington, DC: The National Academies Press. doi: 10.17226/11470.
×

TABLE 3 Logistic Regression Analysis of Use of VA MH/SA Services Among Veterans Who Used Any MH/SA Care, 2001

 

Odds Ratio

95% Confidence Interval

Rank Order

Sociodemographic

Age

<30

0.19*

0.18-0.19

3

30–49

0.38

0.37-0.38

8

50–59

0.86

0.84-0.88

34

60–75

1.21

1.19-1.24

31

>75

1.00

Reference

Race

Black

1.68

1.66-1.70

16

Hispanic

1.29

1.27-1.32

24

Other

1.49

1.47-1.52

19

Whites

1.00

Reference

Gender

Male

1.00

Reference

Female

0.83

0.82-0.85

32

Education

<High School Graduate

1.00

Reference

High School Graduate

1.26

1.24-1.28

28

Post High School Education

0.79

0.77-0.80

25

4 Yr College Degree or Higher

0.73

0.72-0.75

21

Employment

Employed

1.00

Reference

Retired/Disabled

0.90

0.89-0.91

35

Unemployed/Other

1.01

1.00-1.03

37

Marital Status

Married

1.00

Reference

Widowed

0.32

0.31-0.33

6

Separated/Divorced

1.41

1.39-1.42

20

Never Married

1.76

1.74-1.79

15

Income

<$20,000

1.00

Reference

$20,000–$40,000

0.79

0.78-0.80

27

>$40,000

0.37

0.37-0.38

7

Income Information Missing

0.50

0.49-0.51

10

Insurance

Medicaid or Medicare

0.79

0.78-0.80

29

Private

0.24

0.24-0.25

4

Medigap

0.53

0.52-0.54

13

Military Related (for example, CHAMPUS)

0.48

0.48-0.49

9

Other Government Insurance (for example, Indian Health Service)

3.43

3.35-3.50

5

Suggested Citation:"Appendix C: Mental and Substance Use Health for Veterans: Experience with Performance Evaluation in the Department of Veterans Affairs." Institute of Medicine. 2006. Improving the Quality of Health Care for Mental and Substance-Use Conditions. Washington, DC: The National Academies Press. doi: 10.17226/11470.
×

 

Odds Ratio

95% Confidence Interval

Rank Order

Military Experience

Active Duty

0–2 years

1.00

Reference

3–5 years

1.27

1.26-1.28

26

5 years or more

1.36

1.34-1.38

22

Served in a Combat or War Zone

1.01

1.00-1.02

38

Exposed to Dead or Wounded

1.19

1.18-1.20

33

Prisoner of War

1.34

1.29-1.39

23

Health Status

Self-Reported Health Status

Fair-Poor

1.85

1.82-1.87

14

Good

1.95

1.93-1.98

11

Excellent-Very Good

1.00

Reference

Any Health Service Use For

Alcohol or Drugs

1.02

1.00-1.03

36

PTSD

1.50

1.48-1.51

18

Mental Health

1.25

1.24-1.26

30

Service Connected

Not Service Connected

1.00

Reference

<50%

7.08

6.99-7.16

2

>50%

16.56

16.28-16.85

1

Mental health interferes with work or activities at all

0.64

0.64-0.65

17

Proxy Answered Questions

1.93

1.88-1.99

12

*All odds ratios were significant at p <.0001 except for an employment status of unemployed or other, which was not significant. (Combat p value is .503 and use of mental health services for drugs and alcohol p value is .0325)

Suggested Citation:"Appendix C: Mental and Substance Use Health for Veterans: Experience with Performance Evaluation in the Department of Veterans Affairs." Institute of Medicine. 2006. Improving the Quality of Health Care for Mental and Substance-Use Conditions. Washington, DC: The National Academies Press. doi: 10.17226/11470.
×

TABLE 4 Veterans Treated for Primary or Secondary Mental Health (MH) Diagnosis in the VHA in FY 2003, by Specialty Clinic Treatment

Diagnosis

(1) Total with any MH Dx

(2) Percent of total by Dx/SC

Primary MH Diagnosis*

(8) Secondary MH Dx only

(9) Percent with secondary Dx only

(3) Total with Primary MH Dx/SC

(4) Treated in Specialty MH Clinic

(5) Percent by Dx/SC

(6) Treated only in non-MH clinic

(7) Percent Treated in Specialty Clinic

Total

1,218,327

100%

930,098

709,432

100%

220,666

76%

288,229

24%

Dysthymia

495,519

41%

360,511

302,070

43%

58,441

84%

135,008

27%

PTSD

242,850

20%

227,448

216,970

31%

10,478

95%

15,402

6%

Anxiety disorder

238,191

20%

183,062

155,108

22%

27,954

85%

55,129

23%

Major depressive disorder

182,927

15%

177,500

173,010

24%

4,490

97%

5,427

3%

Schizophrenia

100,054

8%

95,733

91,568

13%

4,165

96%

4,321

4%

Adjustment disorder

79,103

6%

73,439

67,674

10%

5,765

92%

5,664

7%

Bipolar disorder

76,702

6%

73,202

70,383

10%

2,819

96%

3,500

5%

Other psychosis

61,985

5%

53,764

39,502

6%

14,262

73%

8,221

13%

Dementia

60,269

5%

38,871

23,047

3%

15,824

59%

21,398

36%

Personality disorder

36,620

3%

35,450

34,427

5%

1,023

97%

1,170

3%

Other psychiatric disorder

274,209

23%

194,649

138,118

19%

56,531

71%

79,560

29%

Substance abuse

266,358

22%

214,251

188,252

27%

25,999

88%

52,107

20%

Alcohol

209,930

17%

166,005

144,854

20%

21,151

87%

43,925

21%

Drug

132,412

11%

121,743

114,994

16%

6,749

94%

10,669

8%

Dually diagnosed (MH and SA)

221,288

18%

211,021

187,808

26%

23,213

89%

10,267

5%

Service connected (1–100%)

462,069

38%

384,133

320,831

45%

63,302

84%

77,936

17%

<=50%

238,690

20%

190,236

150,669

21%

39,567

79%

48,454

20%

>50%<100%

120,818

10%

102,600

88,391

12%

14,209

86%

18,218

15%

100%

102,561

8%

91,297

81,771

12%

9,526

90%

11,264

11%

Total diagnoses/veteran

1.8

 

 

2.2

 

1.0

 

1.4

 

*The primary diagnosis is the principal condition treated during the episode of care.

NOTE: MH = mental health, Dx = diagnosis, SA = substance abuse, SC = service connected, PTSD = posttraumatic stress disorder

Suggested Citation:"Appendix C: Mental and Substance Use Health for Veterans: Experience with Performance Evaluation in the Department of Veterans Affairs." Institute of Medicine. 2006. Improving the Quality of Health Care for Mental and Substance-Use Conditions. Washington, DC: The National Academies Press. doi: 10.17226/11470.
×

TABLE 5 Workload of Specialized VA Mental Health Programs (FY 2003)

Program

Outpatient/Community

Patients Treated

Visits/Pt/Yr.

Total Visits

Outpatient Substance Abusea

115,954

25.8

2,991,613

Mental Health Intensive Case Management (ACT)

3,961

59.3

234,887

Day Hospital

6,218

27.4

170,373

Day Treatment Center

14,389

39.7

571,243

PTSD Clinical Teams

72,849

8.9

645,895

Community Residential Care

9,436

8.4

79,262

Health Care for Homeless Veteransb

61,123

4.1

252,642

Psychosocial Rehabilitationc

38,302

33.5

1,283,117

Total

283,930

17.4

4,945,917

Inpatient/Residential

Episodes of Care

Days/Ep.

Total Days

General Psychiatry Inpatient

87,002

12.8

1,113,626

Psychosocial Residential Rehabilitation and Treatmentd

12,771

40.1

512,117

Inpatient Substance Abuse

5,763

7.4

42,646

Specialized PTSD Programs

4,302

38.8

166,908

Homeless Veterans Grant and Per Diem Program

10,982

92.6

1,016,933

Health Care for Homeless Veterans Residential Contracts

5,430

52.8

286,599

Domiciliary Care for Homeless Veterans

5,156

110.3

568,707

Total

126,250

24.9

3,138,829

aIncludes methadone maintenance and substance day treatment

bIncludes supported housing, homeless outreach, and case management

cIncludes Compensated Work Therapy, Incentive Therapy, Vocational Assistance

dIncludes residential programs for PTSD, substance abuse, general psychiatry, and vocational rehabilitation

Suggested Citation:"Appendix C: Mental and Substance Use Health for Veterans: Experience with Performance Evaluation in the Department of Veterans Affairs." Institute of Medicine. 2006. Improving the Quality of Health Care for Mental and Substance-Use Conditions. Washington, DC: The National Academies Press. doi: 10.17226/11470.
×

Appendix

Performance Measures Used by the Northeast Program Evaluation Center in the Evaluation and Monitoring of VA Mental Health Programs (Data from FY 2002)

  1. Health Care for Homeless Veterans (HCHV) Program and Domiciliary Care for Homeless Veterans (DCHV) Programs

  2. Compensated Work Therapy (CWT) Program and Compensated Work Therapy /Transitional Residence (CWT/TR) Program

  3. PTSD Performance Monitors and Outcome Measures

  4. Mental Health Intensive Case Management

  5. Performance Measures from the National Mental Health Program Performance Monitoring System

  6. Adherence to PORT Pharmacotherapy Guidelines for Patients with Schizophrenia

  7. Outcomes on the Global Assessment of Functioning (GAF) Scale

I. HEALTH CARE FOR HOMELESS VETERANS (HCHV) PROGRAM AND DOMICILIARY CARE FOR HOMELESS VETERANS (DCHV) PROGRAM

Program Structure

Unique veterans served/stops per clinician (HCHV mean = 147.5 veterans; 619 visits)

Percent of allocated staff slots that are filled (HCHV mean = 96.3%)

Literally homeless veterans seen per clinician (HCHV mean = 90.3)

Per diem cost (HCHV mean = $37.67)

Annual turnover rate1 (DCHV mean = 3.3)

Process Measures

Patient Characteristics

Percent strictly homeless (living outdoors/shelter) (HCHV mean = 67.8%; DCHV mean = 34.8%)

Percent with no time homeless (HCHV = 8.1%; DCHV mean = 4.1%)

Percent with a psychiatric disorder, substance abuse problem, or medical illness (HCHV mean = 82%; DCHV mean = 99.8%)

1  

Annual turnover rate is determined by dividing the total number of discharges in the DCHV Program by the number of DCHV operating beds. Average length of stay and occupancy rates will influence a site’s value for annual turnover rate.

Suggested Citation:"Appendix C: Mental and Substance Use Health for Veterans: Experience with Performance Evaluation in the Department of Veterans Affairs." Institute of Medicine. 2006. Improving the Quality of Health Care for Mental and Substance-Use Conditions. Washington, DC: The National Academies Press. doi: 10.17226/11470.
×

Program Participation

Mean days in residential treatment (HCHV mean = 59 days; DCHV mean = 104.1 days)

Percent of successful residential treatment completions (HCHV = 52%; DCHV mean = 69%)

Percent of disciplinary discharges from residential treatment (DCHV mean = 14%)

Percent of premature program departures from residential treatment (DCHV mean = 12%)

Percent contacted through outreach (HCHV = 78%)

Appropriateness for residential treatment (HCHV = 89%)

Outcome Measures

Percent with clinical improvement in alcohol problems (HCHV = 73.1%; DCHV mean = 84% unadjusted)

Percent with clinical improvement in non-substance abuse psychiatric problems(HCHV = 70.7%; DCHV mean = 83% unadjusted)

Percent with clinical improvement with medical problems (HCHV mean = 66.3%, DCHV mean = 88.4%)

Percent discharged to an apartment, room, or house (HCHV mean = 37.9%; DCHV mean = 57.4% unadjusted)

Percent with no housing arrangements after discharge (Supported Housing Program = 21.8%; DCHV mean = 19.3% unadjusted)

Percent discharged with arrangements for full- or part-time employment (HCHV = 49.3%; DCHV mean = 54.5% unadjusted)

II. COMPENSATED WORK THERAPY (CWT) PROGRAM AND COMPENSATED WORK THERAPY/TRANSITIONAL RESIDENCE (CWT/TR) PROGRAM

Process Measures

Program participation

Mean hours worked per week in CWT (CWT mean = 26.1 hr/wk; CWT/TR mean = 33.0)

Mean days in residential treatment (CWT/TR mean = 192 days)

Percent of successful completions (CWT mean = 51.4%; CWT/TR mean = 56.1%)

Percent of disciplinary discharges (CWT mean = 15.4%; CWT/TR mean = 26.8%)

Suggested Citation:"Appendix C: Mental and Substance Use Health for Veterans: Experience with Performance Evaluation in the Department of Veterans Affairs." Institute of Medicine. 2006. Improving the Quality of Health Care for Mental and Substance-Use Conditions. Washington, DC: The National Academies Press. doi: 10.17226/11470.
×

Veteran satisfaction in the residential treatment environment2 (CWT/TR Community Oriented Program Evaluation Scale (COPES) index mean = 2.95)

Veteran satisfaction in the therapeutic work environment3 (CWT/TR Ward Environment Scale (WES) mean = 6.22)

Outcome Measures

Mean work improvement score4 (CWT mean = 1.57)

Percent with clinical improvement in alcohol problems (CWT mean = 63.6% unadjusted)

Percent with clinical improvement in drug problems (CWT mean = 63% unadjusted)

Percent with clinical improvement in nonsubstance abuse psychiatric problems(CWT mean = 47.3% unadjusted)

Percent with clinical improvement with medical problems (CWT mean = 35.2% unadjusted)

Percent competitively employed after discharge (CWT mean = 41.3%)

Percent unemployed after discharge (CWT mean = 24.8%)

Percent employment status unknown after discharge (CWT mean = 14.6%)

Percent of veterans relocated and reinterviewed 3 months after discharge (CWT/TR mean = 55.8%)

Mean Addiction Severity Index (ASI) index for alcohol problems 3 months after discharge (CWT/TR mean = 0.08 unadjusted)

Mean ASI index for drug problems 3 months after discharge (CWT/TR mean = 0.03 unadjusted)

Mean ASI index for psychiatric problems 3 months after discharge (CWT/TR mean = 0.18 unadjusted)

Mean days competitively employed past month at 3 months after discharge (CWT/TR mean = 11 days unadjusted)

Mean days housed past 90 days at 3 months after discharge (CWT/TR mean = 71.2 days unadjusted)

III. PTSD PERFORMANCE MONITORS AND OUTCOME MEASURES

Outpatient Programs (Specialized PTSD Outpatient Programs)

Patient Characteristics

War zone service (87%)

DD 214 (Discharge Certification form) service validation (65%)

2  

Range equals 0–4.

3  

Range equals 0–9.

4  

Range equals 0–2.

Suggested Citation:"Appendix C: Mental and Substance Use Health for Veterans: Experience with Performance Evaluation in the Department of Veterans Affairs." Institute of Medicine. 2006. Improving the Quality of Health Care for Mental and Substance-Use Conditions. Washington, DC: The National Academies Press. doi: 10.17226/11470.
×

PTSD clinical diagnosis (82%)

Substance abuse diagnosis (41%)

Prior psychiatric treatment (74%)

Prior specialized PTSD treatment (22%)

Workload

Number of visits per filled Full Time Employee Equivalents (FTEE) (1,160)

Number of veterans treated per filled FTEE (88)

Costs

Direct costs per visit ($73)

Direct costs per capita ($958)

Outpatient Care Measures (All VA PTSD treatment, specialized and non-specialized)

Service Utilization and Continuity of Care Six Months Following Inpatient Index Stay

Any outpatient stop in 6 months after discharge (DC) (92.0%)

Any outpatient stop in 30 days after DC (72.0%)

Days to first outpatient stop in 6 months after discharge (23.7)

Number of stops in 6 months among those with any stops (18.6)

Continuity: Bi-months (2 month intervals) with two stops (2.5)

(Next measure applies only to those with a dual diagnosis (PTSD and SA)

At least 1 Psychiatric and 1 SA outpatient (OP) stop in 6 months after DC (14.4%)

Continuity of Care Among Outpatients with PTSD Diagnosis

Number of outpatient stops (15.5)

Number of days with outpatient stops (11.9)

Continuity: bi-months with 2 stops (2.61)

Continuity: months with any MH visit of six months (4.17)

Dropout (6 months with no O/P visit) 13%

Continuity of care index (.57)

Modified Continuity Index (MCI) (.80)

Inpatient/Residential Programs (Specialized PTSD programs)

Patient Characteristics

(Same as for Outpatient programs)

Costs

Direct costs per diem ($136)

Direct costs per capita ($4,662)

Suggested Citation:"Appendix C: Mental and Substance Use Health for Veterans: Experience with Performance Evaluation in the Department of Veterans Affairs." Institute of Medicine. 2006. Improving the Quality of Health Care for Mental and Substance-Use Conditions. Washington, DC: The National Academies Press. doi: 10.17226/11470.
×
Outcomes (4 months after discharge)5

PTSD symptoms

Short Mississippi Scale (−5.6%)

NEPEC PTSD Scale (−6.7%)

Substance abuse

ASI Composite for Alcohol problems (−23.5%)

ASI Composite for Drugs problems (−14.3%)

Violence

NVVRS Scale (−38.8%)

Work

Number of days worked for pay (+1.8%)

Satisfaction with treatment

Client Satisfaction Scale (Attkisson et al.) (15.6)

All PTSD Inpatient Care (General and Specialized Programs)

Bed days six months after DC (5.6)

Number of admissions 6 months after DC (.45)

Percentage readmitted within 14 days (5.0%)

Percentage readmitted within 30 days (8.0%)

Percentage readmitted within 180 days (30.0%)

Days to readmission first year after discharge (74.8)

IV. MENTAL HEALTH INTENSIVE CASE MANAGEMENT (MHICM)

Program Structure

Percent allocated FTEE that are filled (84%)

Availability of appropriate medical support (89%)

Availability of appropriate nursing support (96%)

Unfilled FTEE lagged greater than 6 months (43% of teams)

Caseload size (average = 15.4)(should be less than 15)

Appropriateness of Admissions

Hospitalized 30 days or more

Hospitalized more than 30 days in the previous year (82%)

Diagnosis of psychotic illness (78%)

GAF < 50 at admission

5  

Negative values on symptom measures represent improvement (i.e., declining symptoms)

Suggested Citation:"Appendix C: Mental and Substance Use Health for Veterans: Experience with Performance Evaluation in the Department of Veterans Affairs." Institute of Medicine. 2006. Improving the Quality of Health Care for Mental and Substance-Use Conditions. Washington, DC: The National Academies Press. doi: 10.17226/11470.
×
Treatment Process

Clients terminated

Intensity (greater than 1 hour per week)(66.7%)

Services provided in the community 60% of the time (87%)

Face-to-face contacts per week (1.45)

Seen for rehabilitation (36.1%)

Improvement in Therapeutic Alliance (10.2%)

ACT Fidelity Score (4.0)

Outcomes6

Change in inpatient days (6 months before entry to 6 months after) (−47 days; −73%)

Brief Psychiatric Rating Scale (BPRS, Symptoms)(−3.92; −10%)

Brief Symptom Inventory (BSI, Symptoms)(−.22; −11%)

Global Assessment of Functioning (GAF)(−2.01; −5%)

Instrumental Activities of Daily Living (IADL)(+.95, 2%)

Lehman Quality of Life Question (+2.8, 11%)

Housing Independence (+.43, 15%)

Satisfaction with VA Mental Health Services (+1.54, 17.6%)

Satisfaction with MHICM (+.62, 21%)

V. PERFORMANCE MEASURES FROM THE NATIONAL MENTAL HEALTH PROGRAM PERFORMANCE MONITORING SYSTEM

Population Coverage

Proportion of All U.S. Veterans who received VA MH services (2.5%)

Proportion of Veterans Service Connected for Mental Illness who received VA MH services (42.2%)

Proportion of low-income Non Service Connected Veterans who received VA MH services (7.0%)

Proportion of Female Veterans who received VA MH services (3.1%)

Inpatient Care Measures

Bed days six months after DC (6.47)

Number of admissions 6 months after DC (.55)

Percentage readmitted within 14 days (6.9%)

Percentage readmitted within 30 days (11.5%)

6  

Negative values on symptom measures represent improvement (i.e., declining symptoms).

Suggested Citation:"Appendix C: Mental and Substance Use Health for Veterans: Experience with Performance Evaluation in the Department of Veterans Affairs." Institute of Medicine. 2006. Improving the Quality of Health Care for Mental and Substance-Use Conditions. Washington, DC: The National Academies Press. doi: 10.17226/11470.
×

Percentage readmitted within 180 days (32.1%)

Days to readmission first year after discharge (64.7%)

Outpatient Care Measures

Service Utilization and Continuity of Care Six Months Following Inpatient Index Stay

Any outpatient stop in 6 months after DC (82.1%)

Any outpatient stop in 30 days after DC (63.3%)

Days to first outpatient stop in 6 months after discharge (24.9)

Number of stops in 6 months among those with any stops (25.8)

Continuity: Bi-months with two stops (2.16)

Any medical outpatient stop in 6 months after DC (82.8%)

Days to 1st medical OP stop in 6 months after DC (36.6)

Number of OP medical stops in 6 months among those with any stops (10.6)

(Next 4 measures apply to those with a dual diagnosis (Psyc. and SA)

At least 1 Psyc. and 1 SA OP stop in 6 months after DC (21.2%)

At least 3 Psyc. and 3 SA OP stop in 6 months after DC (17.3%)

Continuity: bi-months with two stops (2.17)

Number of Psyc. and SA visits among those with any stops (25.89)

Continuity of Care among Outpatients with Psychotic Diagnoses

Number of outpatient stops (16.8)

Number of days with outpatient stops (12.5)

Continuity: bi-months with 2 stops (2.56)

Continuity: months with any MH visit of 6 months (3.98)

Dropout (6 months with no OP visit) 15%

Continuity of care index (.56)

Modified MCI (.80)

Inpatient Satisfaction Measures

Coordination of Care (.72)

Provision of Information (.66)

Timeliness/Access to Care (.61)

Courtesy (.66)

Emotional Support (.63)

Respect for Patient Preferences (.71)

Family Involvement (.54)

Physical Care (.62)

Transition Home (.62)

General Satisfaction (.53)

Suggested Citation:"Appendix C: Mental and Substance Use Health for Veterans: Experience with Performance Evaluation in the Department of Veterans Affairs." Institute of Medicine. 2006. Improving the Quality of Health Care for Mental and Substance-Use Conditions. Washington, DC: The National Academies Press. doi: 10.17226/11470.
×

VI. ADHERENCE TO PORT PHARMACOTHERAPY GUIDELINES FOR PATIENTS WITH SCHIZOPHRENIA

Percentage of patients receiving oral antipsychotic medication (81%).

Percentage of veterans dosed higher than recommended guidelines (12.3%)

Percentage of veterans dosed lower than recommended guidelines (28.6%)

Percentage of patients receiving polypharmacy (two antipsychotic medications)(8.1%)

VII. OUTCOMES ON THE GLOBAL ASSESSMENT OF FUNCTIONING (GAF) SCALE

Improvement after inpatient discharge

Change from inpatient GAF to last outpatient GAF in first six months after discharge (4.9)

Change from inpatient GAF to the last outpatient GAF of the fiscal year (6.4)

Improvement after during outpatient treatment

Change from first outpatient GAF to last outpatient GAF in the next 6 months (0.46)

Change from first outpatient GAF to last outpatient GAF of the fiscal year (0.33)

Change from first outpatient GAF in the second 6 months of the fiscal year to the last outpatient GAF of the fiscal year (0.38)

Among newly admitted outpatient veterans (those with no outpatient mental health visits in the first 3 months of the fiscal year) the change from first outpatient GAF to last outpatient GAF of the fiscal year (1.9)

Suggested Citation:"Appendix C: Mental and Substance Use Health for Veterans: Experience with Performance Evaluation in the Department of Veterans Affairs." Institute of Medicine. 2006. Improving the Quality of Health Care for Mental and Substance-Use Conditions. Washington, DC: The National Academies Press. doi: 10.17226/11470.
×
Page 423
Suggested Citation:"Appendix C: Mental and Substance Use Health for Veterans: Experience with Performance Evaluation in the Department of Veterans Affairs." Institute of Medicine. 2006. Improving the Quality of Health Care for Mental and Substance-Use Conditions. Washington, DC: The National Academies Press. doi: 10.17226/11470.
×
Page 424
Suggested Citation:"Appendix C: Mental and Substance Use Health for Veterans: Experience with Performance Evaluation in the Department of Veterans Affairs." Institute of Medicine. 2006. Improving the Quality of Health Care for Mental and Substance-Use Conditions. Washington, DC: The National Academies Press. doi: 10.17226/11470.
×
Page 425
Suggested Citation:"Appendix C: Mental and Substance Use Health for Veterans: Experience with Performance Evaluation in the Department of Veterans Affairs." Institute of Medicine. 2006. Improving the Quality of Health Care for Mental and Substance-Use Conditions. Washington, DC: The National Academies Press. doi: 10.17226/11470.
×
Page 426
Suggested Citation:"Appendix C: Mental and Substance Use Health for Veterans: Experience with Performance Evaluation in the Department of Veterans Affairs." Institute of Medicine. 2006. Improving the Quality of Health Care for Mental and Substance-Use Conditions. Washington, DC: The National Academies Press. doi: 10.17226/11470.
×
Page 427
Suggested Citation:"Appendix C: Mental and Substance Use Health for Veterans: Experience with Performance Evaluation in the Department of Veterans Affairs." Institute of Medicine. 2006. Improving the Quality of Health Care for Mental and Substance-Use Conditions. Washington, DC: The National Academies Press. doi: 10.17226/11470.
×
Page 428
Suggested Citation:"Appendix C: Mental and Substance Use Health for Veterans: Experience with Performance Evaluation in the Department of Veterans Affairs." Institute of Medicine. 2006. Improving the Quality of Health Care for Mental and Substance-Use Conditions. Washington, DC: The National Academies Press. doi: 10.17226/11470.
×
Page 429
Suggested Citation:"Appendix C: Mental and Substance Use Health for Veterans: Experience with Performance Evaluation in the Department of Veterans Affairs." Institute of Medicine. 2006. Improving the Quality of Health Care for Mental and Substance-Use Conditions. Washington, DC: The National Academies Press. doi: 10.17226/11470.
×
Page 430
Suggested Citation:"Appendix C: Mental and Substance Use Health for Veterans: Experience with Performance Evaluation in the Department of Veterans Affairs." Institute of Medicine. 2006. Improving the Quality of Health Care for Mental and Substance-Use Conditions. Washington, DC: The National Academies Press. doi: 10.17226/11470.
×
Page 431
Suggested Citation:"Appendix C: Mental and Substance Use Health for Veterans: Experience with Performance Evaluation in the Department of Veterans Affairs." Institute of Medicine. 2006. Improving the Quality of Health Care for Mental and Substance-Use Conditions. Washington, DC: The National Academies Press. doi: 10.17226/11470.
×
Page 432
Suggested Citation:"Appendix C: Mental and Substance Use Health for Veterans: Experience with Performance Evaluation in the Department of Veterans Affairs." Institute of Medicine. 2006. Improving the Quality of Health Care for Mental and Substance-Use Conditions. Washington, DC: The National Academies Press. doi: 10.17226/11470.
×
Page 433
Suggested Citation:"Appendix C: Mental and Substance Use Health for Veterans: Experience with Performance Evaluation in the Department of Veterans Affairs." Institute of Medicine. 2006. Improving the Quality of Health Care for Mental and Substance-Use Conditions. Washington, DC: The National Academies Press. doi: 10.17226/11470.
×
Page 434
Suggested Citation:"Appendix C: Mental and Substance Use Health for Veterans: Experience with Performance Evaluation in the Department of Veterans Affairs." Institute of Medicine. 2006. Improving the Quality of Health Care for Mental and Substance-Use Conditions. Washington, DC: The National Academies Press. doi: 10.17226/11470.
×
Page 435
Suggested Citation:"Appendix C: Mental and Substance Use Health for Veterans: Experience with Performance Evaluation in the Department of Veterans Affairs." Institute of Medicine. 2006. Improving the Quality of Health Care for Mental and Substance-Use Conditions. Washington, DC: The National Academies Press. doi: 10.17226/11470.
×
Page 436
Suggested Citation:"Appendix C: Mental and Substance Use Health for Veterans: Experience with Performance Evaluation in the Department of Veterans Affairs." Institute of Medicine. 2006. Improving the Quality of Health Care for Mental and Substance-Use Conditions. Washington, DC: The National Academies Press. doi: 10.17226/11470.
×
Page 437
Suggested Citation:"Appendix C: Mental and Substance Use Health for Veterans: Experience with Performance Evaluation in the Department of Veterans Affairs." Institute of Medicine. 2006. Improving the Quality of Health Care for Mental and Substance-Use Conditions. Washington, DC: The National Academies Press. doi: 10.17226/11470.
×
Page 438
Suggested Citation:"Appendix C: Mental and Substance Use Health for Veterans: Experience with Performance Evaluation in the Department of Veterans Affairs." Institute of Medicine. 2006. Improving the Quality of Health Care for Mental and Substance-Use Conditions. Washington, DC: The National Academies Press. doi: 10.17226/11470.
×
Page 439
Suggested Citation:"Appendix C: Mental and Substance Use Health for Veterans: Experience with Performance Evaluation in the Department of Veterans Affairs." Institute of Medicine. 2006. Improving the Quality of Health Care for Mental and Substance-Use Conditions. Washington, DC: The National Academies Press. doi: 10.17226/11470.
×
Page 440
Suggested Citation:"Appendix C: Mental and Substance Use Health for Veterans: Experience with Performance Evaluation in the Department of Veterans Affairs." Institute of Medicine. 2006. Improving the Quality of Health Care for Mental and Substance-Use Conditions. Washington, DC: The National Academies Press. doi: 10.17226/11470.
×
Page 441
Suggested Citation:"Appendix C: Mental and Substance Use Health for Veterans: Experience with Performance Evaluation in the Department of Veterans Affairs." Institute of Medicine. 2006. Improving the Quality of Health Care for Mental and Substance-Use Conditions. Washington, DC: The National Academies Press. doi: 10.17226/11470.
×
Page 442
Suggested Citation:"Appendix C: Mental and Substance Use Health for Veterans: Experience with Performance Evaluation in the Department of Veterans Affairs." Institute of Medicine. 2006. Improving the Quality of Health Care for Mental and Substance-Use Conditions. Washington, DC: The National Academies Press. doi: 10.17226/11470.
×
Page 443
Suggested Citation:"Appendix C: Mental and Substance Use Health for Veterans: Experience with Performance Evaluation in the Department of Veterans Affairs." Institute of Medicine. 2006. Improving the Quality of Health Care for Mental and Substance-Use Conditions. Washington, DC: The National Academies Press. doi: 10.17226/11470.
×
Page 444
Suggested Citation:"Appendix C: Mental and Substance Use Health for Veterans: Experience with Performance Evaluation in the Department of Veterans Affairs." Institute of Medicine. 2006. Improving the Quality of Health Care for Mental and Substance-Use Conditions. Washington, DC: The National Academies Press. doi: 10.17226/11470.
×
Page 445
Suggested Citation:"Appendix C: Mental and Substance Use Health for Veterans: Experience with Performance Evaluation in the Department of Veterans Affairs." Institute of Medicine. 2006. Improving the Quality of Health Care for Mental and Substance-Use Conditions. Washington, DC: The National Academies Press. doi: 10.17226/11470.
×
Page 446
Suggested Citation:"Appendix C: Mental and Substance Use Health for Veterans: Experience with Performance Evaluation in the Department of Veterans Affairs." Institute of Medicine. 2006. Improving the Quality of Health Care for Mental and Substance-Use Conditions. Washington, DC: The National Academies Press. doi: 10.17226/11470.
×
Page 447
Suggested Citation:"Appendix C: Mental and Substance Use Health for Veterans: Experience with Performance Evaluation in the Department of Veterans Affairs." Institute of Medicine. 2006. Improving the Quality of Health Care for Mental and Substance-Use Conditions. Washington, DC: The National Academies Press. doi: 10.17226/11470.
×
Page 448
Suggested Citation:"Appendix C: Mental and Substance Use Health for Veterans: Experience with Performance Evaluation in the Department of Veterans Affairs." Institute of Medicine. 2006. Improving the Quality of Health Care for Mental and Substance-Use Conditions. Washington, DC: The National Academies Press. doi: 10.17226/11470.
×
Page 449
Suggested Citation:"Appendix C: Mental and Substance Use Health for Veterans: Experience with Performance Evaluation in the Department of Veterans Affairs." Institute of Medicine. 2006. Improving the Quality of Health Care for Mental and Substance-Use Conditions. Washington, DC: The National Academies Press. doi: 10.17226/11470.
×
Page 450
Suggested Citation:"Appendix C: Mental and Substance Use Health for Veterans: Experience with Performance Evaluation in the Department of Veterans Affairs." Institute of Medicine. 2006. Improving the Quality of Health Care for Mental and Substance-Use Conditions. Washington, DC: The National Academies Press. doi: 10.17226/11470.
×
Page 451
Suggested Citation:"Appendix C: Mental and Substance Use Health for Veterans: Experience with Performance Evaluation in the Department of Veterans Affairs." Institute of Medicine. 2006. Improving the Quality of Health Care for Mental and Substance-Use Conditions. Washington, DC: The National Academies Press. doi: 10.17226/11470.
×
Page 452
Suggested Citation:"Appendix C: Mental and Substance Use Health for Veterans: Experience with Performance Evaluation in the Department of Veterans Affairs." Institute of Medicine. 2006. Improving the Quality of Health Care for Mental and Substance-Use Conditions. Washington, DC: The National Academies Press. doi: 10.17226/11470.
×
Page 453
Suggested Citation:"Appendix C: Mental and Substance Use Health for Veterans: Experience with Performance Evaluation in the Department of Veterans Affairs." Institute of Medicine. 2006. Improving the Quality of Health Care for Mental and Substance-Use Conditions. Washington, DC: The National Academies Press. doi: 10.17226/11470.
×
Page 454
Suggested Citation:"Appendix C: Mental and Substance Use Health for Veterans: Experience with Performance Evaluation in the Department of Veterans Affairs." Institute of Medicine. 2006. Improving the Quality of Health Care for Mental and Substance-Use Conditions. Washington, DC: The National Academies Press. doi: 10.17226/11470.
×
Page 455
Suggested Citation:"Appendix C: Mental and Substance Use Health for Veterans: Experience with Performance Evaluation in the Department of Veterans Affairs." Institute of Medicine. 2006. Improving the Quality of Health Care for Mental and Substance-Use Conditions. Washington, DC: The National Academies Press. doi: 10.17226/11470.
×
Page 456
Suggested Citation:"Appendix C: Mental and Substance Use Health for Veterans: Experience with Performance Evaluation in the Department of Veterans Affairs." Institute of Medicine. 2006. Improving the Quality of Health Care for Mental and Substance-Use Conditions. Washington, DC: The National Academies Press. doi: 10.17226/11470.
×
Page 457
Suggested Citation:"Appendix C: Mental and Substance Use Health for Veterans: Experience with Performance Evaluation in the Department of Veterans Affairs." Institute of Medicine. 2006. Improving the Quality of Health Care for Mental and Substance-Use Conditions. Washington, DC: The National Academies Press. doi: 10.17226/11470.
×
Page 458
Suggested Citation:"Appendix C: Mental and Substance Use Health for Veterans: Experience with Performance Evaluation in the Department of Veterans Affairs." Institute of Medicine. 2006. Improving the Quality of Health Care for Mental and Substance-Use Conditions. Washington, DC: The National Academies Press. doi: 10.17226/11470.
×
Page 459
Suggested Citation:"Appendix C: Mental and Substance Use Health for Veterans: Experience with Performance Evaluation in the Department of Veterans Affairs." Institute of Medicine. 2006. Improving the Quality of Health Care for Mental and Substance-Use Conditions. Washington, DC: The National Academies Press. doi: 10.17226/11470.
×
Page 460
Suggested Citation:"Appendix C: Mental and Substance Use Health for Veterans: Experience with Performance Evaluation in the Department of Veterans Affairs." Institute of Medicine. 2006. Improving the Quality of Health Care for Mental and Substance-Use Conditions. Washington, DC: The National Academies Press. doi: 10.17226/11470.
×
Page 461
Suggested Citation:"Appendix C: Mental and Substance Use Health for Veterans: Experience with Performance Evaluation in the Department of Veterans Affairs." Institute of Medicine. 2006. Improving the Quality of Health Care for Mental and Substance-Use Conditions. Washington, DC: The National Academies Press. doi: 10.17226/11470.
×
Page 462
Suggested Citation:"Appendix C: Mental and Substance Use Health for Veterans: Experience with Performance Evaluation in the Department of Veterans Affairs." Institute of Medicine. 2006. Improving the Quality of Health Care for Mental and Substance-Use Conditions. Washington, DC: The National Academies Press. doi: 10.17226/11470.
×
Page 463
Suggested Citation:"Appendix C: Mental and Substance Use Health for Veterans: Experience with Performance Evaluation in the Department of Veterans Affairs." Institute of Medicine. 2006. Improving the Quality of Health Care for Mental and Substance-Use Conditions. Washington, DC: The National Academies Press. doi: 10.17226/11470.
×
Page 464
Suggested Citation:"Appendix C: Mental and Substance Use Health for Veterans: Experience with Performance Evaluation in the Department of Veterans Affairs." Institute of Medicine. 2006. Improving the Quality of Health Care for Mental and Substance-Use Conditions. Washington, DC: The National Academies Press. doi: 10.17226/11470.
×
Page 465
Suggested Citation:"Appendix C: Mental and Substance Use Health for Veterans: Experience with Performance Evaluation in the Department of Veterans Affairs." Institute of Medicine. 2006. Improving the Quality of Health Care for Mental and Substance-Use Conditions. Washington, DC: The National Academies Press. doi: 10.17226/11470.
×
Page 466
Suggested Citation:"Appendix C: Mental and Substance Use Health for Veterans: Experience with Performance Evaluation in the Department of Veterans Affairs." Institute of Medicine. 2006. Improving the Quality of Health Care for Mental and Substance-Use Conditions. Washington, DC: The National Academies Press. doi: 10.17226/11470.
×
Page 467
Suggested Citation:"Appendix C: Mental and Substance Use Health for Veterans: Experience with Performance Evaluation in the Department of Veterans Affairs." Institute of Medicine. 2006. Improving the Quality of Health Care for Mental and Substance-Use Conditions. Washington, DC: The National Academies Press. doi: 10.17226/11470.
×
Page 468
Suggested Citation:"Appendix C: Mental and Substance Use Health for Veterans: Experience with Performance Evaluation in the Department of Veterans Affairs." Institute of Medicine. 2006. Improving the Quality of Health Care for Mental and Substance-Use Conditions. Washington, DC: The National Academies Press. doi: 10.17226/11470.
×
Page 469
Suggested Citation:"Appendix C: Mental and Substance Use Health for Veterans: Experience with Performance Evaluation in the Department of Veterans Affairs." Institute of Medicine. 2006. Improving the Quality of Health Care for Mental and Substance-Use Conditions. Washington, DC: The National Academies Press. doi: 10.17226/11470.
×
Page 470
Suggested Citation:"Appendix C: Mental and Substance Use Health for Veterans: Experience with Performance Evaluation in the Department of Veterans Affairs." Institute of Medicine. 2006. Improving the Quality of Health Care for Mental and Substance-Use Conditions. Washington, DC: The National Academies Press. doi: 10.17226/11470.
×
Page 471
Suggested Citation:"Appendix C: Mental and Substance Use Health for Veterans: Experience with Performance Evaluation in the Department of Veterans Affairs." Institute of Medicine. 2006. Improving the Quality of Health Care for Mental and Substance-Use Conditions. Washington, DC: The National Academies Press. doi: 10.17226/11470.
×
Page 472
Suggested Citation:"Appendix C: Mental and Substance Use Health for Veterans: Experience with Performance Evaluation in the Department of Veterans Affairs." Institute of Medicine. 2006. Improving the Quality of Health Care for Mental and Substance-Use Conditions. Washington, DC: The National Academies Press. doi: 10.17226/11470.
×
Page 473
Suggested Citation:"Appendix C: Mental and Substance Use Health for Veterans: Experience with Performance Evaluation in the Department of Veterans Affairs." Institute of Medicine. 2006. Improving the Quality of Health Care for Mental and Substance-Use Conditions. Washington, DC: The National Academies Press. doi: 10.17226/11470.
×
Page 474
Suggested Citation:"Appendix C: Mental and Substance Use Health for Veterans: Experience with Performance Evaluation in the Department of Veterans Affairs." Institute of Medicine. 2006. Improving the Quality of Health Care for Mental and Substance-Use Conditions. Washington, DC: The National Academies Press. doi: 10.17226/11470.
×
Page 475
Suggested Citation:"Appendix C: Mental and Substance Use Health for Veterans: Experience with Performance Evaluation in the Department of Veterans Affairs." Institute of Medicine. 2006. Improving the Quality of Health Care for Mental and Substance-Use Conditions. Washington, DC: The National Academies Press. doi: 10.17226/11470.
×
Page 476
Suggested Citation:"Appendix C: Mental and Substance Use Health for Veterans: Experience with Performance Evaluation in the Department of Veterans Affairs." Institute of Medicine. 2006. Improving the Quality of Health Care for Mental and Substance-Use Conditions. Washington, DC: The National Academies Press. doi: 10.17226/11470.
×
Page 477
Suggested Citation:"Appendix C: Mental and Substance Use Health for Veterans: Experience with Performance Evaluation in the Department of Veterans Affairs." Institute of Medicine. 2006. Improving the Quality of Health Care for Mental and Substance-Use Conditions. Washington, DC: The National Academies Press. doi: 10.17226/11470.
×
Page 478
Suggested Citation:"Appendix C: Mental and Substance Use Health for Veterans: Experience with Performance Evaluation in the Department of Veterans Affairs." Institute of Medicine. 2006. Improving the Quality of Health Care for Mental and Substance-Use Conditions. Washington, DC: The National Academies Press. doi: 10.17226/11470.
×
Page 479
Suggested Citation:"Appendix C: Mental and Substance Use Health for Veterans: Experience with Performance Evaluation in the Department of Veterans Affairs." Institute of Medicine. 2006. Improving the Quality of Health Care for Mental and Substance-Use Conditions. Washington, DC: The National Academies Press. doi: 10.17226/11470.
×
Page 480
Suggested Citation:"Appendix C: Mental and Substance Use Health for Veterans: Experience with Performance Evaluation in the Department of Veterans Affairs." Institute of Medicine. 2006. Improving the Quality of Health Care for Mental and Substance-Use Conditions. Washington, DC: The National Academies Press. doi: 10.17226/11470.
×
Page 481
Suggested Citation:"Appendix C: Mental and Substance Use Health for Veterans: Experience with Performance Evaluation in the Department of Veterans Affairs." Institute of Medicine. 2006. Improving the Quality of Health Care for Mental and Substance-Use Conditions. Washington, DC: The National Academies Press. doi: 10.17226/11470.
×
Page 482
Next: Index »
Improving the Quality of Health Care for Mental and Substance-Use Conditions Get This Book
×
Buy Hardback | $59.95 Buy Ebook | $47.99
MyNAP members save 10% online.
Login or Register to save!
Download Free PDF

Each year, more than 33 million Americans receive health care for mental or substance-use conditions, or both. Together, mental and substance-use illnesses are the leading cause of death and disability for women, the highest for men ages 15-44, and the second highest for all men. Effective treatments exist, but services are frequently fragmented and, as with general health care, there are barriers that prevent many from receiving these treatments as designed or at all. The consequences of this are serious—for these individuals and their families; their employers and the workforce; for the nation’s economy; as well as the education, welfare, and justice systems. Improving the Quality of Health Care for Mental and Substance-Use Conditions examines the distinctive characteristics of health care for mental and substance-use conditions, including payment, benefit coverage, and regulatory issues, as well as health care organization and delivery issues. This new volume in the Quality Chasm series puts forth an agenda for improving the quality of this care based on this analysis. Patients and their families, primary health care providers, specialty mental health and substance-use treatment providers, health care organizations, health plans, purchasers of group health care, and all involved in health care for mental and substance–use conditions will benefit from this guide to achieving better care.

  1. ×

    Welcome to OpenBook!

    You're looking at OpenBook, NAP.edu's online reading room since 1999. Based on feedback from you, our users, we've made some improvements that make it easier than ever to read thousands of publications on our website.

    Do you want to take a quick tour of the OpenBook's features?

    No Thanks Take a Tour »
  2. ×

    Show this book's table of contents, where you can jump to any chapter by name.

    « Back Next »
  3. ×

    ...or use these buttons to go back to the previous chapter or skip to the next one.

    « Back Next »
  4. ×

    Jump up to the previous page or down to the next one. Also, you can type in a page number and press Enter to go directly to that page in the book.

    « Back Next »
  5. ×

    Switch between the Original Pages, where you can read the report as it appeared in print, and Text Pages for the web version, where you can highlight and search the text.

    « Back Next »
  6. ×

    To search the entire text of this book, type in your search term here and press Enter.

    « Back Next »
  7. ×

    Share a link to this book page on your preferred social network or via email.

    « Back Next »
  8. ×

    View our suggested citation for this chapter.

    « Back Next »
  9. ×

    Ready to take your reading offline? Click here to buy this book in print or download it as a free PDF, if available.

    « Back Next »
Stay Connected!