Controlling Costs and Changing Patient Care?: The Role of Utilization Management (1989)

Page 13

1
Utilization Management: Introduction and Definitions

Prior to having the cholecystectomy recommended by her physician, Greta Harrison calls an 800 telephone number to notify the organization that does utilization management for her employer. That organization gets in touch with the surgeon's office to discuss various aspects of the care that is proposed for her. Is hospitalization necessary or can the surgery be done as an ambulatory procedure? How long will the patient need to be in the hospital? In this case, the reviewer agrees that inpatient care is clearly appropriate but questions the plan to admit the patient two days prior to surgery. Since the patient lives in the same town as the hospital and can easily have preoperative tests performed on an outpatient basis, the surgeon agrees to admit her on the day of the surgery.

After Michael Travers is admitted to the hospital following a myocardial infarction, the hospital—aware of his benefit plan's requirements—notifies the appropriate utilization management organization. The length of stay is discussed, but no explicit target date for discharge is set. However, the hospital is then called every third day by the organization, which evaluates information about the patient's need for further hospitalization. The calls continue until Mr. Travers, who has a difficult recovery, is improved enough to be discharged to his home. The physician has not had to adjust the treatment plan but feels irritated at the "red tape" involved. And Mr. Travers has worried on some occasions that payment for part of his hospital stay might be denied.

With their daughter depending on a ventilator to breathe and receiving other hospital care for muscular dystrophy, the parents of Patty Simon are contacted by a case manager for the insurance company that covers the family. The question is whether they and their physician would like to explore arrangements for home care, which is possible in this case but considerably more complex than usual. With the parents' and physician's cooperation, the case manager works out a plan for transfer that includes assessment of the home's wiring (which is adequate for the equipment), provision for two shifts of home nursing care every day, and purchase of appropriate medical equipment and supplies. This requires some expenditures not normally covered by the benefit plan, but the employer agrees with the insurer to make an exception in this case because the arrangements will not only be less costly than hospital care but will also improve the quality of life for the family.

Page 14

With great rapidity and relatively little public awareness, a significant change has taken place in the way some decisions are made about a patient's medical care. Many decisions like those just described, once the exclusive province of the doctor and patient, now have to be examined in advance by an external reviewer, someone who is accountable to an employer, insurer, health maintenance organization (HMO), preferred provider organization (PPO), or other entity responsible for paying all or most of the cost of the care. Depending upon the circumstances, this outside party may be involved in discussions about whether a service is needed, how treatment will be provided, and where care will occur.

This preliminary Institute of Medicine (IOM) report describes the nature of this change in medical decision-making and assesses its impact on patients, providers, and purchasers of medical services. It focuses on the utilization management efforts of the private sector, which provides health benefits for most Americans under age 65.1

Prior review of proposed medical care is not entirely new in the 1980s. Review organizations for Medicare were performing some preadmission review in the 1970s, and some private payers made limited use of the technique even earlier. However, widespread application of this approach to managing health care utilization is a phenomenon of the 1980s.

A survey conducted in 1983 reported that only 14 percent of corporate benefit plans required prior approval of nonemergency admissions to hospitals (Equitable Life Assurance Society of the United States, 1983). By 1988, another survey found 95 of 100 large firms had such programs (Corporate Health Strategies, 1988). Perhaps half to three-quarters of employees nationwide are now covered by such programs, up from only 5 percent in 1984 (Foster Higgins, 1987; Gabel et al., 1988).

What accounts for this rapid spread of utilization management through external assessments of the need for proposed medical services? The most obvious factor is rapidly rising health care costs. Purchasers' search for effective ways to limit their financial liability for health benefits stems directly from their belief that costs are out of control.

The trends responsible for this view are painfully familiar to everyone concerned with health care financing. In 1987, the latest year for which statistics are available, total spending on health care reached an estimated $500 billion, up from $234 billion just 5 years earlier (Levit and Freeland, 1988). This spending has been increasing at a rate considerably above the rate of general inflation (Table 1-1), and the share of the gross national product attributed to health services went from 5.9 percent in 1965 to 11.1

1 Public programs have been the subject of several reports in recent years (for example, General Accounting Office, 1983, 1988a, 1988b; Health Care Financing Administration, 1979; Physician Payment Review Commission, 1988, 1989, and Project HOPE, 1987).

Page 15

percent in 1987. Spending for health care by business as a percentage of the gross private domestic product grew from 1.1 percent in 1965 to 3.4 percent in 1987 (Levit et al., 1989).

High health care costs for employers have been cited as one factor impairing American competitiveness in world markets and a reason why many small firms do not provide health benefits for workers. In 1987, spending for health care by business equaled about 6 percent of total labor compensation compared with about 2 percent in 1965 (Figure 1-1) (Levit et al., 1989). A recent survey of nearly 800 employers of all sizes reported average premium increases from 1987 to 1988 of 11 percent for conventional insurance plans and between 8 and 10 percent for HMOs (Gabel et al., 1989). Another survey cited average increases from 1987 to 1988 of 14 percent for employers with insured programs and 25 percent for employers with self-insured programs (Foster Higgins, 1989). Companies that self-insure assume all or most of the financial risk of paying for covered medical services used by employees and their dependents instead of paying an outside insurance to accept that risk. In the private insurance sector, many commercial insurers, Blue Cross and Blue Shield plans, and HMOs have seen significant underwriting losses—$3.6 billion for commercial carriers and $1.1 billion for Blue Cross and Blue Shield plans in 1988 (Donahue, 1989). Some commercial insurers, for example, Kemper, Provident Mutual, Allstate (for large groups only), and Transamerica Occidental, are withdrawing from the group health insurance market (Meyer and Page, 1988).

To the dismay over rising health care costs has been added a growing perception that much medical care is unnecessary and sometimes harmful. The studies that have contributed to this perception have also produced some optimism that external review of physician practice decisions could detect unnecessary care, influence physician behavior, and reduce costs without jeopardizing access to needed services (Eisenberg, 1986; Schwartz,

Page 16

Figure 1-1
Expenditures by private industry for health services and 
supplies as a percent of total labor compensation, 1965-1987.

Source: Levit et al. (1989, p. 9).

1984; Wennberg, 1984; Wennberg et al., 1977). In addition, experience has suggested that review of some care prospectively—prior to its provision— would be more palatable and effective than retrospective review has been. This set of perceptions and expectations is, in essence, the hypothesis of utilization management, a hypothesis of interest to patients, practitioners, purchasers, and policymakers.

The IOM Committee on Utilization Management by Third Parties has examined the utilization management hypothesis by asking several questions.

• How effective is utilization management in limiting utilization and containing costs?

• Are there unintended positive and negative consequences of bringing an outside party into the process of making decisions about patient care?

• Are utilization management organizations and purchasers sufficiently accountable for their actions, or are new forms of oversight, perhaps government regulation, needed?

Page 17

• What are the responsibilities of health care providers and patients for the appropriate use of health services?

The committee's investigatory approach has been described in the preface. Chapters 2 through 5 discuss the committee's findings about why utilization management has become so widespread, how utilization management actually operates and appears to be evolving, and what is known about its effects. In Chapter 6, the committee assesses the current status of utilization management, including its strengths and shortcomings, and recommends near-term and longer-range actions that could help utilization management realize its objectives of controlling costs and reducing inappropriate services without undermining patient access to needed care.

What Is Utilization Management?

In its study of utilization management, the committee found that the term has no single, well-accepted definition. As with the labels cost containment and managed care, different people may mean different things by the same term. In this report, the committee considers utilization management as a set of techniques used by or on behalf of purchasers of health care benefits to manage health care costs by influencing patient care decision-making through case-by-case assessments of the appropriateness of care prior to its provision.

Three points about the committee's focus are worth underscoring. First, the committee examines methods that rely on case-by-case assessments of care. Second, the focus is on review prior to the provision of services. Third, this report stresses actions taken to reduce costs for third-party purchasers of care. The first characteristic distinguishes utilization management from methods that analyze aggregate utilization patterns to identify potential problems or that rely on across-the-board limits on health care benefits that take no account of individual patient characteristics. The second characteristic differentiates utilization management from the retrospective review of claims or medical records submitted after care has been provided. The third characteristic directs attention to purchaser-sponsored—rather than provider-sponsored—utilization management efforts, except when providers explicitly share the financial risk with purchasers of care, as they do in HMOs.

The dominant utilization management strategy is prior review of proposed medical services, which includes several related techniques. A second, more focused, strategy is high-cost case management (see Table 1-2).

Prior Review

Prior review provides advance evaluation of whether medical services proposed for a specific person conform to provisions of health plans that

Page 18

limit coverage to medically necessary care.2 Most prior review programs include an integrated set of review steps, not all of which will apply to any single patient. The focus may be on the site of care, the timing or duration of care, or the need for a specific procedure or other service.

The first point of assessment, often called preadmission review, may occur before an elective hospital admission. This is what Greta Harrison and her physician experienced in one of the vignettes that opened this chapter. In this case, the review did not challenge the need for the procedure itself or the need for hospital care, but it did challenge the proposed admission 2 days before surgery. The terms preservice review and preprocedure review are sometimes used to indicate that the focus of review is the need for a procedure, regardless of whether it is to be performed on an inpatient or an outpatient basis.

For emergency or urgent admissions to the hospital when prior review is not reasonable or feasible, admission review may be required within 24 to 72 hours after hospitalization to check the appropriateness of the admission as early as possible. The vignette describing Mr. Travers involved this technique as well as continued-stay review or concurrent review, which assesses the length of stay for both urgent and nonurgent admissions. Reviewers may press for timely discharge planning by hospital staff and, in some instances, assist in identifying and arranging appropriate alternatives to inpatient care.

In addition, a patient may be required to get a second opinion on the need for certain proposed treatments from a practitioner other than the patient's physician. Increasingly, preadmission review or preservice review is used to screen patients so that referrals for second opinions are focused on patients for whom the clinical indications for a service are dubious.

To encourage patients covered by a health plan to cooperate in the

2 Medical necessity is another term that is used differently by different people in different contexts. Some use it generally to cover assessments of the site and duration of care as well as the clinical need for a particular procedure, whereas others use it only in the latter sense. Those who use the term more restrictively tend to apply the term appropriateness to the former assessments. For a discussion of legal interpretations of medical necessity, see the paper by William A. Helvestine in Appendix A of this report.

Page 19

prior review process, a financial penalty, such as higher cost-sharing, may apply when individuals fail to obtain necessary certifications. Chapter 3 provides more details about the mechanisms of prior review.

Although terms like prior review, predetermination, precertification, and prior authorization of benefits are often used interchangeably, the approval of benefits in advance of service provision may be contingent rather than final. For example, if a retrospective claims review suggests that the information on which the predetermination was based was seriously flawed, payment of a claim may be denied upon further investigation. Or if a utilization management firm does not have access to the details of the benefit plan for a group, it might authorize services not covered by the contract. A review of claims prior to payment might then result in denial of benefits. Since this latter practice usually makes patients unhappy, many utilization management firms try to consider restrictions in a client's health plan in their determinations. Retrospective denials of claims following prior certification appear to be rare, as are refusals to preauthorize services.

High-Cost Case Management

High-cost case management—also called large case management, medical case management, catastrophic case management, or individual benefits management—focuses on the relatively few beneficiaries in any group who have generated or are likely to generate very high expenditures. This small percentage of individuals—perhaps 1 to 7 percent of a group—may account for 30 to 60 percent of the group's total costs. For the United States as a whole in 1980, 1 percent of the population accounted for 29 percent of total health care spending (Berk et al., 1988).

Case management for individuals with high-cost illnesses is similar to other forms of social and health case management, in that it involves assessing a person's needs and personal circumstances and then planning, arranging, and coordinating the recommended services. It differs in its targets, those very expensive cases for which specialized attention may encourage appropriate but less costly alternative forms of treatment.

In contrast to prior review programs, high-cost case management programs are usually voluntary, with no penalties for patient failure to become involved in the process or comply with its recommendations. (In the third vignette, Patty Simon's parents could have refused the alternative course of care suggested for her.) In addition, more effort is generally devoted to reviewing the patient's particular condition and circumstances and exploring, even arranging, alternative modes of treatment. Finally, exceptions to limitations in benefit contracts may be authorized in advance if this will permit appropriate but less expensive care. For instance, additional home nursing benefits may be arranged so that an individual can avoid further

Page 20

hospitalization. In unusual cases, benefits may be provided for other than health care services, such as construction of a wheelchair ramp or rewiring a patient's home, if these expenditures will allow home care or self-care to be substituted for institutional services at a lower total cost. (The assessment of the wiring in Patty Simon's home would have been covered in this fashion.)

Retrospective Utilization Review

Utilization management techniques, particularly prior review methods, attempt to overcome the disadvantages and unhappiness associated with retrospective review and denial of claims after services have already been provided. Retrospective claims and medical record reviews can, however, support and reinforce utilization management by

• monitoring the accuracy of information provided during prior review and identifying problem areas,

• examining claims that are unsuitable for predetermination (generally those with high volume and low unit costs), and

• analyzing patterns of practitioner or institutional care for use in provider education programs and selective contracting arrangements.

Retrospective utilization review methods have a longer history of general application than do prospective methods (Blum et al., 1977; Congressional Budget Office, 1979, 1981; Institute of Medicine, 1976; Law, 1974). Its strengths and weaknesses have been scrutinized in a number of studies before this one and are not explicitly considered in this report. However, constraints on retrospective review have been a key stimulus for the development of prior review methods. Many of the concerns raised by the committee about the clinical soundness of review criteria, the fairness of procedures, and other matters described apply to both prospective and retrospective reviews.

Other Cost-Containment Methods

The techniques of prior review and high-cost case management are but a subset of the cost-containment methods that can influence decisions about patient care. Other methods, some of which are discussed in Chapter 2 and Appendix B, include the following:

• benefit design (including patient cost-sharing and coverage exclusions), consumer education, and other approaches that shape patient demand for care;

• financial incentives (for example, capitation or bonuses) that are designed to reward physicians or institutions for providing less costly care;

Page 21

• contracts with health care practitioners and institutions that establish limits on payment for care provided to health plan enrollees;

• use of gatekeeping, triaging, and other devices to manage patient flow to specialists and expensive services; and

• physician education and feedback on standards of care and patterns of practice.

Utilization management shares with the last four strategies a recognition of the physician's central role as the player-manager of the health care team who is responsible for organizing and directing the production process and providing some of the productive input (Eisenberg, 1986). The different strategies for influencing decisions about patient care, however, vary in their emphasis or reliance on different models of control (such as professional self-regulation, informed consumerism, or prudent purchasing), their techniques of influence (such as education, financial incentives, peer pressure, or external oversight), and the parties involved (that is, patients, primary care practitioners, or specialists).

As will be described in Chapter 2, different strategies for cost containment have been tried, abandoned, and revived as third-party financing of health care has expanded. This history reflects both the difficulties of the task and an appreciation that there is no single solution to problems of health care costs, quality, or access. Many strategies have a place, each of which has different strengths and weaknesses and each of which needs monitoring and adjustment as circumstances change and people adapt to various attempts to shape their behavior.

Two Notes of Caution

Obstacles To Evaluation

This report laments the limited evidence on utilization management and calls repeatedly for more and better assessments. Nonetheless, the committee is well aware that sound evaluation of utilization management programs faces several obstacles. Some are intrinsic to the research problem, some reflect common organizational behaviors, and some involve particular pressures faced by market-driven organizations. Rigorous evaluation also tends to be quite expensive. In Appendix B of this report, the commissioned paper by Joan B. Trauner notes that evidence about the impact of physician financial incentives on patient care decisions and quality of care is also quite limited.

Intrinsic Conceptual and Methodological Problems

A number of problems in evaluating utilization management and other cost-containment programs are predictable difficulties faced, to one degree

Page 22

or another, in much social and evaluation research (Eddy and Billings, 1988; Wennberg, 1987). One such problem is that there are no uniformly accepted and applied rules for measuring health care utilization or adjusting data for differences in the characteristics of groups being compared. Other methodological difficulties involve (1) data quality and availability; (2) definitions and measurements of program characteristics, group characteristics, outcomes, and other variables; (3) projections of what would have happened without the interventions; and (4) generalizations to other programs and settings.

Common Behavioral Biases Against Evaluation

Under this heading come obstacles to systematic evaluation that are typical of organizations whether they be public or private, for-profit or not-for-profit, big or small (Eddy and Billings, 1988; Hatry et al., 1973; March and Simon, 1958; Suchman, 1967). They include preferences for

• action over evaluation, for example, developing, selling, and running a program rather than seeing if it works;

• quick payoff rather than long-term products or results;

• easy rather than difficult actions (for example, using data on inputs and procedures that are simpler to collect rather than data on outputs or outcomes);

• compelling anecdotes, consensus, or tradition over careful and complex analyses; and

• positive rather than negative results.

In addition, faced with limited resources, managers are frequently reluctant to allocate funds for evaluation instead of wages and benefits, shareholder dividends, or other activities. The committee has no information about what utilization management firms spend on evaluation (for internal use or for clients) or how much different employers invest in systematically assessing the impact of prior review or other cost-containment strategies.3

Competition and Evaluation

The normal individual and organizational biases against systematic evaluation may be both mitigated and intensified in competitive environ-

3The private sector is not alone in providing meager resources for program evaluation. The utilization and quality review components of Medicare's peer review organization (PRO) program have not been very rigorously examined (General Accounting Office, 1988a; Physician Payment Review Commission, 1988). The Health Care Financing Administration does have performance standards for PROs, but they tend to emphasize process rather than outcome and tend to involve measures of impact that are more appropriate for ongoing monitoring rather than systematic evaluation of the review techniques.

Page 23

ments. Certainly, competition can be a powerful stimulus for internal evaluation of how well a product is working and what makes it work better. Also, clients of utilization management organizations have a strong interest in obtaining reports on results and in shifting their business to other firms if they cannot get such reports.

Balanced against these forces are several threats posed by evaluation. Most obviously, an evaluation may be negative and thereby reduce a firm's chances for retaining clients or winning new clients.4 Moreover, when an evaluation is publicly available, a firm's competitors gain information that could help them build a case to inform potential clients that the competitor could provide better results or, at least, better reports. Further, evaluations of utilization management programs may provide competitors with statistical norms or even provider-specific information that would not be readily available to them otherwise. Likewise, if firms that invest in relatively sophisticated research and development reveal their work, they may give a free ride for competitors to copy or build on the resulting review criteria, analytic methodologies, or other products. In a new and rapidly evolving industry, this can seem a significant issue for more experienced organizations.

Forces Behind Rising Health Care Costs

The Committee on Utilization Management by Third Parties also recognizes that the forces behind rising health care costs are exceptionally strong and difficult to constrain through moderate means. Many believe that, for the foreseeable future, health care costs will continue to increase faster than costs in the rest of the economy.

• Clinical judgments about the value of treatment for various categories of patients are changing as new treatments or new evidence of treatment impact emerges. For example, women who underwent mastectomy for breast cancer and had no evidence that the cancer had spread were until recently not expected to benefit from chemotherapy, but some new analyses suggest such treatment does increase survival rates. It also increases initial treatment costs (Early Breast Cancer Trialists' Collaborative Group, 1988). Recent guidelines for the use of mammography screening could greatly expand the amount of such screening but some professional sources question whether the guidelines are clinically warranted (McIlrath, 1989).

4Even when the reported results were positive, the committee encountered considerable reluctance by review organizations to have their analyses published.

Page 24

• New tests may reduce diagnostic uncertainty but not add any information that aids in treatment decision-making (Kassirer, 1989). Advances in screening techniques may catch individuals much earlier in the course of disease and reduce the numbers who will receive later expensive treatments. The question is, will the costs of screening and early treatment offset the savings? Will real survival rates increase? Researchers involved with cancer point to methods under development to screen for very early traces of dozens of different kinds of cancer, not all of which are more successfully treated if they are detected earlier.

• The work force and the general population are aging, and the use of both acute-care and long-term-care services is higher for people in the older age groups.

• Between 1980 and 2000, the number of physicians has been projected to increase from 171 to 260 per 100,000 population (Graduate Medical Education National Advisory Committee, 1981; U.S. Department of Health and Human Services, 1985). Whether this will bring a surplus of physicians is a matter for debate (Ginsburg, 1989; Schwartz et al., 1989). Nonetheless, one estimate, now many years out of date, is that every additional physician results in $400,000 in additional yearly expenditures for medical services.

• The concern about the millions of Americans who have no routine health insurance coverage is generating various proposals to protect these individuals through, for example, state-sponsored insurance pools, mandated employer-based insurance, expansions of Medicaid, and universal federal health insurance (Congressional Research Service, 1988). What are the short-term costs (and for whom) of increasing access? What long-term costs and benefits can be expected?

Reducing increases in health care costs such that they are much closer to the level of general inflation would appear to demand radical changes in American health policy, either major restructuring of the financing and delivery systems or major cutbacks through large shifts in costs to patients, severe limitations on patients' choices of hospitals and physicians, and explicit rationing of some technologies for all or some individuals. Society may not be willing to make such changes, particularly in the short run (Curran, 1987). It may continue the search, described in the next chapter, for more moderate strategies to control health care expenditures. Utilization management is one such strategy.

It is an unfortunate reality, however, that most cost-containment strategies eventually disappoint their supporters and evaluators to some degree. Even when these strategies seem to reduce costs initially, trend projections do not appear to show an appreciably lower increase in total costs over the longer term (Prospective Payment Assessment Commission, 1989). Given

Page 25

the effort and optimism it generally takes to commit a corporation or a government to a new program, it is not surprising that excessively high expectations often give way eventually to disillusionment. Unwarranted or excessive negativism can, in turn, be counterproductive and lead to premature abandonment of modest but still helpful strategies.

Cognizant of these hazards, the Committee on Utilization Management by Third Parties has tried to approach its initial evaluation of utilization management with reasonable expectations. To this end, the committee has reviewed the development of third-party financing of health care in the United States and the ways in which various strategies to manage costs have evolved. The next chapter summarizes this review.

References

Berk, Marc L, Monheit, Alan C., and Hagan, Michael M., ''How the U.S. Spent Its Health Care Dollar, 1929-1980,'' Health Affairs, Fall 1988, pp. 46-60.

Blum, John D., Gertman, Paul M., and Rabinow, Jean, PSROs and the Law, Germantown, MD: Aspen Systems Corporation, 1977.

Congressional Budget Office, The Effects of PSROs on Health Care Costs: Current Findings and Future Evaluations, Washington, DC, 1981.

Congressional Budget Office, The Impact of PSROs on Health Care Costs: Update of CBO's 1979 Evaluation, Washington, DC, 1981.

Congressional Research Service, Health Insurance and the Uninsured. Background Data and Analysis, Washington, DC, June 9, 1988.

Corporate Health Strategies, The Health Poll, Fall 1988, p. 1.

Curran, William J., The Health Care Cost-Containment Movement: A Reconsideration, Report of a conference sponsored by Medicine in the Public Interest, Baltimore, MD, March 16 and 17, 1987.

Donahue, Richard, "Health Premiums Soared Past the $100 Billion Mark in 1988," National Underwriter, June 5, 1989, p. 1.

Early Breast Cancer Trialists' Collaborative Group, "Effects of Adjuvant Tamoxifen and of Cytotoxic Therapy on Mortality in Early Breast Cancer," New England Journal of Medicine, December 29, 1988, pp. 1681-1692.

Eddy, David M., and Billings, John, "The Quality of Medical Evidence and Medical Practice," Paper prepared for the National Leadership Conference on Health Care, Washington, DC, 1988.

Eisenberg, John M., Doctors' Decisions and the Cost of Medical Care, Ann Arbor, MI: Health Administration Press, 1986.

Equitable Life Assurance Society of the United States, The Equitable Healthcare Survey: Options for Controlling Costs, conducted by Lou Harris and Associates, Inc., New York, August 1983.

Foster Higgins, Health Care Benefits Survey, New York, 1987.

Foster Higgins, Health Care Benefits Survey, New York, 1989.

Gabel, Jon, DiCarlo, Steven, Fink, Steven, and de Lissovoy, Gregory, "Employer-Sponsored Health Insurance in America," Research Bulletin of the Health Insurance Association of America, Washington, DC, January 1989.

Gabel, Jon, Jajich-Toth, Cindy, de Lissovoy, Oregory, and Cohen, Howard, "The Changing World of Group Health Insurance," Health Affairs, Summer 1988, pp. 48-65.

General Accounting Office, Improving Medicare and Medicaid Systems to Control Payment for Unnecessary Physicians' Services, GAO/HRD-83-16, Washington, DC, February 8, 1983.

Page 26

General Accounting Office, Medicare: Improving Quality of Care Assessment and Assurance, GAO/PEMD-88-10, Washington, DC, May 1988a.

General Accounting Office, Medicare PROs: Extreme Variation in Organizational Structure and Activities, GAO/PEMD-89-7FS, Washington, DC, November 1988b.

Ginzberg, Eli, "Physician Supply in the Year 2000," Health Affairs, Summer 1989, pp. 84-90.

Graduate Medical Education National Advisory Committee, Report to the Secretary, Vol. 1, DHEW Publication No. HRA 81-6510, Washington, DC, 1981.

Hatry, Harry P., Winnier, Richard E., and Risk, Donald M., Practical Program Evaluation for State and Local Governments, Washington, DC, The Urban Institute, 1973.

Health Care Financing Administration, Professional Standards Review Organizations: 1978 Program Evaluation, Washington, DC, U.S. Department of Health, Education, and Welfare, 1979, pp. 34-37.

Health Insurance Association of America, 1986-1987 Source Book of Health Insurance Data, Washington, DC, 1987.

Health Insurance Association of America, 1988 Update: Source Book of Health Insurance Data, Washington, DC, 1988.

Institute of Medicine, Assessing Quality in Health Care, Washington, DC, National Academy of Sciences, 1976.

Kassirer, Jerome P., "Our Stubborn Quest for Diagnostic Certainty," New England Journal of Medicine, June 1, 1989, pp. 1489-1491.

Law, Sylvia, Blue Cross: What Went Wrong?, New Haven, CT, Yale University Press, 1974.

Letsch, Suzanne W., Levit, Katherine R., and Waldo, Daniel R., "National Health Expenditures, 1987," Health Care Financing Review, Winter 1988, pp. 109-122.

Levit, Katharine R., and Freeland, Mark S., "National Medical Care Spending," Health Affairs, Winter 1988, pp. 124-136.

Levit, Katherine R., Freeland, Mark S., and Waldo, Daniel, "Health Spending and Ability to Pay," Health Care Financing Review, Spring 1989, pp. 1-12.

March, James G., and Simon, Herbert A., Organizations, New York, John Wiley & Sons, Inc., 1958.

McIlrath, Sharon, "11 Groups Endorse Mammogram Guidelines," American Medical News, July 14, 1989, pp. 3, 35.

Meyer, Harris, and Page, Leigh, "New Era in Utilization Review," American Medical News, December 9, 1988, pp. 1, 45.

Physician Payment Review Commission, Annual Report to Congress, Washington, DC, March 1988.

Physician Payment Review Commission, Annual Report to Congress, Washington, DC, April 1989.

Project HOPE, A Study of the Preadmission Review Process, Report prepared for the Prospective Payment Assessment Commission, Chevy Chase, MD, November 1987.

Prospective Payment Assessment Commission, Medicare Prospective Payment and the American Health Care System: Report to Congress, Washington, DC, June 1989.

Schwartz, J. Sanford, "The Role of Professional Medical Societies in Reducing Variations," Health Affairs, Summer 1984, pp. 90-101.

Schwartz, William B., Sloan, Frank A., and Mendelson, Daniel N., "Debating the Supply of Physicians: The Authors Respond," Health Affairs, Summer 1989, pp. 91-95.

Suchman, Edward A., Evaluative Research, New York, Russell Sage Foundation, 1967.

U.S. Department of Health and Human Services, Projections of Physician Supply in the U.S., ODAM Report No. 3-85, Washington, DC, Bureau of Health Professions, 1985.

Waldo, Daniel R., Levit, Katherine R., and Lazenby, Helen, "National Health Expenditures," Health Care Financing Review, Fall 1986, pp. 1-21.

Wennberg, John E., Blowers, Lewis, Parker, Robert and Gittelshon, Alan M., "Changes in Tonsillectomy Rates Associated with Feedback and Review," Pediatrics, June 1977, pp. 821-826.

Wennberg, John E., "Dealing with Medical Practice Variations: A Proposal for Action," Health Affairs, Summer 1984, pp. 6-32.

Wennberg, John E., "Use of Claims Data Systems to Evaluate Health Care Outcomes," Journal of the American Medical Association, February 20, 1987, pp. 933-936.