The Artificial Heart: Prototypes, Policies, and Patients (1991)

THE NATIONAL HEART, LUNG, AND BLOOD INSTITUTE (NHLBI) must make numerous decisions in coming years about funding in its artificial heart program. The main purpose of this study is to suggest factors to be considered in these decisions, including whether to continue the development of fully implantable ventricular assist devices (VADs) and total artificial hearts (TAHs). As a framework for the specific research and development topics in the balance of the report, this chapter presents explicit decision-making criteria and describes several methods for applying them.

As discussed further in Chapter 9, the NHLBI artificial heart program is unusual among health care R&D programs in that government funds have provided almost all of the support for developing the long-term, fully implantable devices that are the subject of this study. In making decisions about activities of the artificial heart program, NHLBI and not-for-profit research organizations use approaches and methods very different from those used by industry in allocating funds to R&D programs (a topic discussed briefly at the end of this chapter).

Decisions about the use of funds for research and development occur at various levels in any organization involved with R&D. Figure 3.1 illustrates some of the types of decisions made in NHLBI, especially ones that involve the artificial heart program.

Allocations must be made, in total, to each of the three major programs within NHLBI's scope. In each of the 23 clinical program areas, decisions

FIGURE 3.1 Levels of R&D resource allocation. The asterisk denotes targeted research that may be accomplished either through R01 grants (by a request for applications) or through contracts (by a request for proposals). NHLBI, National Heart, Lung, and Blood Institute; CHD, coronary heart disease; TAH, total artificial heart; VAD, ventricular assist device; R&D, research and development.

are needed about the amounts to be allocated for specific targeted research efforts and for nontargeted, investigator-initiated research. Once a specific targeted allocation is fixed, such as the total for all grants under a particular request for applications (RFA) initiative or for funding of contracts for VAD or TAH development, administrators must decide the specific amount allocated to each grant or contract. In general, two methods can be used in the allocation of R&D funds—decision making that is solely judgment-based or decisions that are aided by quantitative techniques such as cost-

effectiveness analysis (CEA) or by systematic application of explicit criteria.

NHLBI currently relies on peer- or professional-judgment decision making in almost all of its funding-allocation decisions. Using heart and vascular diseases as an example, the NHLBI Cardiology Advisory Committee periodically ranks all proposed new initiatives (RFAs) in priority order. Except for this peer-judgment priority setting for new R&D initiatives, the committee understands that NHLBI does not use quantified methods or assessments of explicit criteria to aid it in deciding about funding allocations at any of the levels indicated on Figure 3.1.

In contrast, in the area of decisions about the scientific merit of specific grants and contracts, NHLBI relies heavily on peer-review judgments that are expressed quantitatively by members of initial review groups (study sections) and ad hoc review committees. Agency officials establish weighted evaluation factors, either general criteria for nontargeted investigatorinitiated grants or factors that are tailored to a specific area for which proposals have been solicited. Examples of evaluation factors include the feasibility of the proposed approach, the adequacy of the experimental design, and the competence of the investigators.

Study section or review committee members assign priority scores to each factor based on scientific and technical merit; the total scores are then converted to percentiles in order to normalize scores across the various groups. Awards are typically made in the order of assigned percentiles.

Traditionally, within NHLBI as in many other organizations, most R&D funding allocation decisions are made on the basis of professional judgment, either individual or collective, about the relative merits of the alternatives under consideration. Rarely is any quantitative approach used, or even a systematic application of specific criteria. General considerations are likely to be implicit in the judgments made, such as these listed in an early draft of Department of Health and Human Services principles for an aborted research planning process:

Investments in areas of health research should be guided by many factors, including research opportunity, burden of illness measures, demographic trends, public perceptions of the relative importance of different health areas, current state of the art, previous investment and return experiences, near term potential for new breakthroughs, problem areas identified through disease surveillance and investigation of disease outbreaks, interrelationships with other research problems, benefits which may accrue by elimination or improved treatment of disease,

the perception of those concerned or involved in dealing with social problems or needs, and a continuing retrospective assessment of health benefits attained through research expenditures. (IOM, 1979, pp. 57-58)

At a more practical level, NHLBI administrators, its National Advisory Council, and other advisory committees are likely to consider any relevant spending allocations in previous years, as well as considering several general factors in making funding decisions and recommendations. These factors include:

• the societal and political importance of the various alternative uses for the available funds;

• the magnitude and nature of the clinical needs in each area;

• the ability to make progress in solving the problem addressed by each area; and

• the research capacity (in such respects as facilities and qualified researchers), as well as the perceived degree of scientific-community professional interest, in particular areas.

Any of the foregoing criteria can be described in some detail, and alternatives can be assessed or at least ranked using them. However, this is rarely done. Further, when decision makers face a choice between basic research and applied R&D, other specific considerations may override general criteria. For instance, those in charge of an R&D effort might assess the nature of the problems that need to be addressed during the R&D work. Are the problems to be solved “basic” ones, or technical and practical ones? If the problem is basic, then basic (but perhaps targeted) research should receive the greater part of the research funds; if the problem is technical, the allocation should favor the particular applied research mechanisms that can best achieve the needed engineering and technical solutions.

Several resource allocation methods have been developed that take into consideration the cost-effectiveness of R&D alternatives. These typically express in dollar terms both the net benefits yielded by developing solutions to specific problems and the projected R&D expenditures needed to achieve each solution. For example, Weinstein (1983) examined the cost-effectiveness of developing and implementing two cancer-prevention research strategies by determining cost-effectiveness ratios that compared the projected benefits of each strategy to their R&D and health care costs.

The calculations also reflected the probability that each R&D effort would accomplish its cancer-prevention goal.

In a similar study, Hatziandreu and colleagues (1988) compared annual spending on research concerning the acquired immunodeficiency syndrome (AIDS), heart disease, cancer, and unintentional injuries (accidents) with the approximate societal cost of each, with costs expressed in several different ways. The study identified current levels of R&D expenditures and related them to such cost measures as deaths caused or years of expected life lost. Such a method could provide valuable information, for example, to Congress or the Office of Management and Budget in making major appropriations or budget decisions.

In a more elaborate consideration of options for vaccine R&D, an Institute of Medicine (IOM) study developed comprehensive estimates of the impact of various contagious diseases (e.g., treatment costs and mortality and morbidity burden) and of the benefits, cost, and probability of successful vaccine development (IOM, 1985). The benefits and costs related to each alternative were combined, using net present value computations for those in future years, resulting in a ranking of the most beneficial R&D efforts to be undertaken.

In addition to using CEA as an aid to decision making, agency administrators can also use a systematic assessment of compliance with explicit criteria. Such a systematic assessment and a CEA can be employed separately or together in a combined approach. Depending on the type of decision, one method or the other may be more desirable.

For decisions that involve broad options, such as allocating funds between basic research and applied R&D or between alternatives that include combinations of programs with little in common, cost-effectiveness analyses or detailed assessments of criteria are not likely to be feasible. If officials explicitly state applicable criteria, however, they improve the likelihood that their judgment-based decisions will be both more reasonable and more consistent over time, even if they do not evaluate alternatives with precision.

In some instances of this nature, detailed, explicit criteria such as those discussed below can be applied. Assessing criteria in detail can serve, for example, as a check against possible arbitrary decisions that may result from pressures brought by groups with a particular interest in the subject under consideration. In other cases, whether for allocations between broad program areas (e.g., “heart” versus “lung”) or between specific target areas within the artificial heart program, using a systematic method will ensure that all relevant factors are considered in the decision-making process.

The committee suggests three methods by which NHLBI administrators and others can be aided in making systematic choices among R&D options. Two are described in the sections that follow; the third, to be discussed in Chapter 6 in connection with this study's cost-effectiveness analysis, deals with the allocation of funds within a single R&D program. The two that follow can be used either alone or in combination.

Industrial firms are able to make decisions about the allocation of their R&D funds among specific programs based on return on investment calculations or net present value analysis. Government agencies, philanthropic foundations, and other groups that support health care R&D seldom experience direct monetary benefit from their research commitments, and thus they cannot use financial or economic analysis alone.

Priority-setting criteria have been developed to identify particular areas of health care or specific technologies that warrant intensive clinical study; see, for example, criteria for clinical conditions to be given high priority for examination by the Public Health Service 's new effectiveness program (IOM, 1989) and priorities for technology assessments to be conducted (IOM, 1990). Such criteria have only limited usefulness, however, in considering R&D funding allocation decisions.

The committee believed a set of explicit criteria tailored for R&D use would be valuable to NHLBI and others, but a search of the literature did not reveal any listing of explicit criteria, with indications of their importance, that can be considered for this particular purpose. Therefore, the committee undertook a fresh start in exploring the question of R&D allocation criteria. Specifically, it developed through staff research and committee discussion, and proposes here, a total of 18 criteria, ranked in the perceived order of their importance for possible use in allocating research funds (see Table 3.1).¹ It felt comfortable doing so because of the wide range of perspectives and broad expertise in health care delivery and policy represented on the committee.

NHLBI can use these ranked criteria as examples of ones that may be

appropriate for its administrators to use in allocating research funds. By suggesting these explicit criteria, the committee hopes to elicit broader discussion, as well as to stimulate NHLBI to use these or other criteria, experimentally at first and then with increasing confidence based on the early experience.

Criteria such as those that follow are difficult to apply, as is stated with respect to a number of them. By attempting to apply these or similar criteria to R&D allocation decisions that arise in day-to-day operations, NHLBI is likely both to achieve greater objectivity and consistency in its decisions and also to make a contribution to the management of R&D by other agencies that administer research.

Health and life expectancy gains. The criterion ranked most important by the committee is the degree to which the R&D program's output will extend anticipated beneficiaries' lives or improve their functioning and health-related quality of life, or both; it was ranked as “very important” by all but 3 of the 17 respondents. This criterion encourages allocation decisions that are based on the expected health benefit to individual patients. Life expectancy gains can be estimated by expert clinicians, and improvement in health-related quality of life can be assessed by using estimates of “before and after” measures as well as by explicitly taking the risks of the new R&D output into account.

Using this criterion in isolation does not relate treatment gains to the costs of achieving them. Further, it requires estimating specific treatment outcomes many years in advance and thus carries an even higher degree of uncertainty than other possible criteria.

Again, just as clinical indications for use may broaden over time, a technology's ability to improve patients' health-related quality of life may also change. Thus, the point in time at which the R&D output's effectiveness² is being measured should govern estimates of both the benefit it confers on the typical patient and the number of persons benefited.

Finally, the health impact of a particular R&D output may sometimes yield benefits of very different types to different groups. For example, a definitive treatment for a particular communicable disease will directly benefit those whose life is prolonged or health improved by the treatment. Additionally, it will have a broader public health impact that can also be estimated: by reducing the size of the diseased population, it also will

reduce the future impact of the disease and its attendant direct and indirect costs.

Number of persons benefited. Another of the more important criteria is the approximate number of persons who will benefit directly from a research output. This criterion is an aggregate measure of the R&D program's potential nationwide significance. Using it with only total numbers of projected patients will not reflect the seriousness of the disease being addressed (e.g., degree of disability it causes) or the extent to which successful research is likely to alleviate the disease's impact on affected individuals. Also, it does not provide for consideration of particular demographic groups (e.g., the poor, older persons) who may or may not benefit disproportionately from the R&D output. It is, therefore, not a criterion that should stand alone, despite its clarity and the high degree of importance with which it is viewed by the committee.

One problem in applying this criterion is the need to estimate the size of a potential user population, which is especially difficult during early stages of R&D. Even with a program in as late a stage of development as the long-term ventricular assist device that is about to undergo clinical trials, projecting indications for use that will be implemented when its routine clinical use begins is difficult. Nonetheless, to apply this criterion, decision makers must develop indications for use, based on the best possible estimates of risks versus benefits, and translate them into numbers of prospective beneficiaries.

In health care, even such a needs-based projection may not reflect two other important considerations. First, at least for costly technologies, actual utilization will depend heavily on the extent to which third-party payers decide to pay for the particular R&D output, as well as on the adequacy of payment rates. This consideration will be particularly important for technologies such as mechanical circulatory support systems (MCSSs), insofar as the Medicare program's decisions are concerned, because end-stage heart disease disproportionately affects older persons.

Second, as has occurred in the cardiovascular arena with coronary artery bypass surgery, coronary angioplasty, and the automatic implantable cardioverter defibrillator, indications for use will often broaden as physicians accept the technology's original clinical applications and begin to use it with other patients. Initial utilization projections will likely become obsolete after a few years of routine use, at least to the extent that third-party payers impose no obstacles to such broadening or “slippage.”

One limitation of assigning this criterion a high degree of importance is that, when relatively few people are affected by a particular disease, specially targeted R&D support may be needed in order to overcome the

effect of this criterion. For example, special legislation was needed to stimulate the development of “orphan” drugs for rare diseases.

Likelihood of R&D success. The committee ranked the probability of R&D success as another of the more important criteria. Quantifying the likelihood that an R&D effort will achieve its goal is difficult, however, especially at an early stage or even before the activity begins. Probability estimates of success can sometimes be included in CEAs, which would make this a more useful criterion.

Usually, the cumulative probability of success should be the basis for applying this criterion to specific R&D alternatives, because the cumulative probability increases steadily over time. Only if criteria are being applied as of a given date should a time-specific probability be used, such as at the end of a program's particular year.

Cost-effectiveness. The fourth of the criteria ranked by the committee as more important is the relative cost-benefit or cost-effectiveness anticipated from the use of each alternative. In an era of enhanced consciousness about health care costs, using anticipated cost-benefit ratios or cost-effectiveness measures (e.g., cost per incremental quality-adjusted life year, or QALY) reflects the relative potential of each R&D output for accomplishing gains in health-related quality of life. As discussed in Chapter 6, cost-benefit analysis (CBA) and cost-effectiveness analysis (CEA) are well recognized as aids to health care decision making; they are particularly valuable if utility measures, assessing the quality of life, are included in the computations.

A method such as CEA also provides a way to reflect costs of alternatives more specifically than may otherwise be possible. A form of treatment that appears to be very costly may, for example, prevent a several-year course of deteriorating health that requires even more costly inpatient care. By explicitly considering the long-term costs if the new technology is notused as well as the costs of using it, CEA presents to the decision maker the net cost of each new technology as well as its benefits.

CBA and CEA studies have limitations, however. Other nonquantifiable factors must often be considered as part of the evaluation process, such as the other criteria suggested in this chapter. Also, a CBA or CEA often requires the analyst to estimate the values of many different variables, some of which may be uncertain. Sensitivity analysis helps to determine how important the uncertainty is to the decision.

To reflect fully the economic impact of an R&D output, the value of possible spin-offs from that effort would also ideally be considered, as a possible offset against total costs. Several MCSS developers have reported successful spin-offs from this R&D (Appendix C; Poirier, 1990). Some of

these have the potential for major improvements to the quality of health care or even potential value in other fields. Without placing some value on possible research spin-offs, evaluating an R&D program's cost-effectiveness would not be complete. Difficulties in forecasting such spin-offs, however, make it unlikely that the choice between R&D options would be affected by differential expectations for spin-off technologies at the time of the R&D decision, unless the options under consideration vary greatly in their amenability to spin-offs.

Committee members evaluated 10 criteria as important in addition to the 4 just discussed in the “more important” range, but applying as many as 14 criteria in comparing a group of R&D proposals is a daunting prospect. Thus, decision makers may wish to limit their use of criteria such as those that follow to perhaps the four or five that are most applicable to the area and alternatives under consideration. All 10 certainly should not be accorded equal weight.

Advancement of knowledge. The highest ranked of the “important” group of criteria is the extent to which a particular R&D program is expected to lead to basic knowledge that is useful in areas other than the one directly aimed at originally. This criterion or goal is often referred to as “advancing knowledge.” It gauges the extent to which targeted research may result in discoveries that are not directly related to the original target, rather than focusing on the basic research itself. The research output may nonetheless be useful in either the alternative under consideration or others.

It is difficult to predict the extent to which an R&D program will produce other useful knowledge. Evaluating options as meeting this criterion will therefore carry considerable uncertainty, especially if the assessment is being done at an early stage of the R&D effort.

The issue of spin-off or spillover discoveries, just discussed in connection with cost-effectiveness analysis, is a related, more concrete example of the value that can result in other fields from targeted research. Still, it may be similarly difficult to evaluate.

Relevance to NHLBI's mission. The relevance of each option to NHLBI's mission is the criterion ranked second highest in the “important” group. Using it may be unproductive, however, because of prior administrative action that, at an early stage, screens out R&D program proposals as irrelevant to NHLBI's mission before they reach the point of formal allocation decisions.

In considering the application of this criterion, NHLBI must recognize

that its mission statement is so broad that it may provide little assessment guidance; this is, unfortunately, true for most large, multifaceted organizations. Still, if the options under review vary considerably in how central or peripheral their goals are to NHLBI 's mission, this will be an important criterion to consider.

Impact on health care costs. The expected impact of each technology on overall health care costs, including the cost of manufacturing and using each alternative R &D output, is another important criterion for allocation decisions. Using the long-term fiscal impact of a particular R&D program as a criterion requires assuming that the research is successful and that the output has come into wide use. Doing so puts R&D decision makers in an unusual position. It may require them, for instance, to confront directly the desirability of applying R&D resources toward achieving an ultimate goal that the United States will be unable to afford or, at a minimum, to consider questions of distributive justice that may arise when the R&D output becomes available. Although manufacturing and service industries routinely consider many aspects of the distribution and sale of their R&D outputs from the outset of planning, similar factors are rarely considered in government and not-for-profit R&D decisions. Distributive justice issues can be considered, however, by NHLBI and other R&D agencies.

Several pitfalls may need to be confronted in applying this criterion:

• Various perspectives (e.g., state, federal, private payer) should be discussed and evaluated thoroughly, because implications of a choice may vary widely by perspective.

• Whenever possible, both direct and indirect costs and benefits associated with the new R&D output should be considered, in order to reflect all offsetting reductions.

• Application of this total-cost criterion, without also considering cost-effectiveness, may stifle innovation. A proposed technology with high projected costs will, in isolation, be viewed adversely by some. In actuality, though, it may be a cost-effective use of resources because of offsetting economic savings or because it provides more substantial health benefits than could be achieved by alternate uses of the resources.

• Long-term health care costs related to a particular R&D proposal are extremely difficult to estimate, making assessments of this criterion uncertain except to the extent that CBA or CEA sensitivity analyses reflect such uncertainties.

Finally, if a technology's use is concentrated within a particular demographic group, projections of the future size of that group should be used in estimating the option's impact on health care costs. The number of older

persons, for instance, will increase greatly in the coming decades. The aggregate cost of applying a technology that is primarily used by older persons is therefore likely to increase at a rate higher than is typical for health care costs generally.

Impact on patients' families. An important criterion that is related to improved life expectancy and health-related quality of life is the effect of the alternative on patients' family members and other caregivers. Its importance will vary depending on the characteristics of each R&D option's goals. If the illness or condition is one in which family members' feelings are likely to have a direct impact on the patient's quality of life, as discussed in Chapter 5, then this criterion may be especially important. Conversely, if family members are typically not involved as caregivers and if the patient's autonomy of decision making about the illness is highly valued, then this criterion should not be applied despite the possible impact on family members. Assessing the effect of an R&D output on family members is even more difficult than it is for patients, particularly in the early stages of an R&D effort, so this criterion should be applied with care.

Nonfinancial impact on health care system. An additional important criterion is the expected impact of each technology on aspects of the health care system (e.g., available beds, personnel) other than the dollar costs of providing the technology. Nonfinancial resources can potentially be a serious constraint on the usefulness of an R&D output; this may be true particularly in the early years of a treatment technology's use as it is applied to a backlog of patients for whom no treatment previously existed. Whether the constraint is one of space—for instance, crowded operating room schedules—or qualified personnel, nonfinancial resource limits may be important with some outputs.

A similar impact may occur when a newly introduced technology has a dramatic effect on the volume of use of other technologies. This occurred, for instance, when thrombolytic therapy was introduced as a treatment for acute myocardial infarction, leading to increased use of coronary angiography and angioplasty (Steinberg et al., 1988).

This criterion thus makes explicit a potential constraint on diffusion that is often not considered but can be important. It is difficult to apply accurately, however, in the early stages of R&D.

Beneficiary demographics. Whether the output will have general applicability or be either applicable or inapplicable to specific age, sex, or ethnic-racial groups is another important criterion. This criterion recognizes the possibility of targeting of R&D resources, one type of which is positive targeting that serves beneficiary groups whose needs are qualitatively the

greatest. Such targeting assumes that need can be assessed in a just and equitable manner; as examples, research on cystic fibrosis or congenital heart disease primarily benefits children and research on sickle cell disease primarily benefits blacks. Conversely, targeting that is undesirable from a distributive justice perspective could also be involved, if it benefits one group (e.g., middle-class white males) disproportionately to the demographics or importance of the particular disease, at the same time ignoring other patients' needs.

Depending on how this criterion is assessed, its application may weight the decision-making process either for or against particular population groups; it must therefore be used with great care. Further, it has the potential to allow political considerations to influence R&D resource allocations, if it yields decisions favoring alternatives that benefit particular groups of individuals who have either emotional appeal via the media or greater perceived political power than others. Assessment weightings under this criterion should be as rational and as relevant as possible to the options being considered.

Ethical considerations. The applicability to a technology of such specific ethical considerations as distributive justice is another of our group of important criteria. Two complexities need to be confronted: (1) when ethical principles as discussed in terms of a single technology conflict with one another, such as in the conflict between providing benefit to individuals and fairly distributing the costs of those benefits within society; and (2) when different ethical considerations affect the R&D alternatives being reviewed. The ethical issues raised by heart transplantation and the TAH are an example. The development of the surgical technique of heart transplantation raised issues about the process for equitably distributing very limited benefits, while development of the TAH raises particular issues of access. (See Chapter 8 for further discussion of ethical considerations pertaining to R &D decisions.)

“Halfway” versus “high” technology. A further important criterion is whether the research is directed at a “halfway” technology (i.e., one that overcomes a disease's impact only after it has fully developed, or postpones death) or at a “high” technology that prevents or cures disease, such as prevention of coronary artery disease by cholesterol-level reduction early in life, before the atherosclerotic process begins.³ By assigning greater weight to high technologies, this criterion would explicitly encourage their devel-

opment, as Lewis Thomas espouses (Thomas, 1972). To the degree that high technologies typically produce highly ranked CEA results, this criterion may, in practice, overlap with cost-effectiveness analysis. Further, the committee did not rank this criterion more highly because it is rarely applicable; relatively few disease entities appear to be susceptible to a high technology, at least in the short term.

Cost-effectiveness related to non-heart disease options. Another criterion is the cost-effectiveness of each R&D output relative to the cost-effectiveness of non-heart disease technologies. In addition to considering the cost-effectiveness of alternative uses of R&D resources within health care, it is also possible, but more difficult, to compare health care research alternatives with R&D programs in other fields. This might be particularly useful in such fields as the environment, where years of life gained, deaths prevented, or added QALYs can be used as a measure in the same manner as in health care. The committee rated this criterion fairly low for two reasons: the general difficulty of comparing such alternatives in quantitative terms, and the belief that such allocation decisions are more appropriately made through the budget process of the administration and Congress.

Extent of existing R&D support. The last of the criteria assessed as “important”—the extent to which industry or other nongovernmental funds already support the particular research (or are expected to do so)—is a useful criterion, when it can be assessed. Application of this criterion minimizes the expenditure of scarce government R&D resources on a goal that can likely be achieved through other funding sources. Still, commitments of private R&D resources can never be considered firm until they occur, so assessing this criterion carries considerable uncertainty until the private support is actually provided.

Time required for implementation. The expected time required for successful implementation of results of the R&D option is another possible criterion. In certain circumstances, it might be a fairly significant one, for instance when considering intensive, time-limited development efforts such as the Small Business Innovation Research program or when the R&D alternatives differ little on other criteria. Usually, however, alternatives that are able to be compared by criteria such as those discussed to this point will not differ greatly in the time required to carry them through to implementation. In contrast, more global programs with goals like “fighting cancer” or “reducing infant mortality” may have widely varying implementation periods.

Likelihood of stimulating private-sector R&D. The extent to which the allocation of NHLBI resources to the particular research is likely to stimulate private-sector research in the same area is related to the previously mentioned criterion concerning the existence of nongovernment R&D support. This criterion recognizes the appeal of leveraging the effect of scarce government resources, in situations in which a multiplier effect can be assumed. It suffers, however, from the same problem as the previous criterion, namely that private R&D commitments can never be considered firm in advance, leading to uncertainty when it is applied.

Application of this criterion requires the government to begin decreasing its R&D support at some point, in the hope that the private sector has been adequately stimulated, but determining that point is difficult. Finally, government-supported research can usually be controlled and monitored more closely than private R&D, in such respects as the steering and data review committees that NHLBI utilizes to oversee clinical trials in this area. Tight control is less likely to occur with private research, as evidenced by the ethical and patient-related criticisms of the mid-1980s cases involving implantation of the Jarvik-7 heart.

Concern about technology misapplication. Although it is an issue that the committee believes is important in relation to MCSSs (see Chapter 7), the degree of concern that an R&D output will be misapplied generally is not viewed in the “important” category. This concern relates to the potential effect of a technology 's misuse in terms of quality of care and health care expenditures, as well as to the cost and ability to maximize appropriate use. To the extent that a particular technology is misapplied, resulting for instance in utilization several times greater than anticipated, levels of precision in assessing and applying other criteria considered here may be substantially in error.

Impact on U.S. global standing. The expected impact of each allocation decision on the nation's global standing, especially with high-cost technologies involving long-term research, is another possible criterion, but one that was rated as lowest in importance by committee members. This criterion is illustrated by the following questions: If a particular R&D output is not developed in the United States, will other countries do so? If so, will both the balance of payments and the nation's position of leadership in health care technology suffer?

This criterion reflects an important general national concern, but it is not directly relevant to health care and thus is accorded little weight by the committee. Still, if most long-term production of mechanical circulatory support devices eventually comes from foreign manufacturers and few controls are placed on the devices' use, the aggregate cost of importing

these devices would have considerable impact on the nation's balance of payments deficit. The technology's overall impact on U.S. health care expenditures will be similar, however, whether the devices are manufactured domestically or overseas.

A formula. Because all four of the “more important” criteria reflect the number of persons served by an R&D output, their benefit from it, and the cost and likelihood of realizing that benefit, decision makers can combine these criteria into an index of the cost-effectiveness (C/E) of the R&D investment. The rest of this section discusses how that might be done and offers a relatively simple formula that would help to quantify the process, by creating a C/E ratio that expresses the value of each alternative research proposal as the R&D cost per a measure of net health outcome added as a result of applying the specific R&D project's output. This situation was not directly involved in the committee's study, however, so this formula has not been used in this report.

The numerator of the C/E ratio to be calculated is the amortized annual cost of the R&D (amortized over the expected life of the technology, which can for simplicity be regarded as infinite). The denominator of the C/E ratio is the annual net health benefit to the population benefited, less the opportunity cost of the health care resources needed to implement the technology. The opportunity cost represents the amount of health benefit forgone by not using health care resources in other areas. The denominator may be measured in units of health outcome, such as QALYs. Thus, the C/E ratio is calculated as follows:

The numerator, or annualized R&D cost, is

C_R = r× K× (1 + r)^t,

where C_R is the annualized R&D cost, ris the real interest rate, Kis the capitalized R&D cost, and tis the delay time until implementation of the technology, which is assumed to continue indefinitely from that time forward.

The denominator is

E = P× (Q− C/R) × N,

where Eis the expected annual effectiveness, Pis the probability of R&D success, Qis the number of QALYs gained per person benefited, Cis the net treatment cost per person benefited, Ris the cost per QALY of alternative uses of health resources, and Nis the number of persons benefited per year.

The value of Rin the denominator, which represents the cost of gaining one QALY by other health care interventions, can be set initially at $50,000 (approximately the cost per QALY for renal dialysis). The basic $50,000 value can then be varied by way of sensitivity analysis, simultaneously for all programs that are being compared. By recomputing all C/E ratios using, perhaps, $10,000 and then $100,000 per QALY, the effect (if any) of these variations in life-extension valuation on the relative rankings of the R&D options will be revealed. In general, lower values of Rimply greater opportunity costs for resources diverted to the new technology and, therefore, tend to result in reduced net effectiveness (E) when the formula is applied to technologies with high costs.

To simplify the calculations, it is convenient to calculate the number of patients benefited (N), annual benefit per patient (Q), and the annual treatment cost (C) on an incidence basis. Thus, Nis the number of patients initiating treatment per year (e.g., receiving a TAH implant), Qis the present value of the total number of QALYs gained per incident case (e.g., compared with not receiving a TAH), and Cis the present value of the treatment cost per case (e.g., also compared with no TAH). This approach avoids the troublesome problem of converting what may be an uneven time stream of treatment costs into an annual equivalent.

The compound interest factor, (1 + r)^t, accounts for the fact that alternative R&D programs under consideration may yield results over very different time frames. This factor (using a typical interest rate such as r = 0.05) corrects for the lead time for each R&D effort. It is important that the R&D cost first be capitalized to present value (K) before it is annualized over the useful life of the technology, which is assumed to extend from year tto infinity.

Applying this ratio to each R&D program under consideration will result in approximate costs per QALY gained that can be roughly compared. The R&D alternatives can then be ranked and evaluated in conjunction with unquantifiable criteria and considerations.

The formula discussed here is a simplification of one developed for what is perhaps the only comprehensive cost-effectiveness analysis of public-sector R&D alternatives in health care ever published, an IOM study of vaccine R&D options for the National Institutes of Health (IOM, 1985). Weinstein (1983) offers further examples of applying a similar formula. The full methodology used in the vaccine R&D study will be particularly useful when some of the R&D alternatives produce technologies that are projected to yield net economic savings (C< 0) and others are projected to yield health benefits at a net economic cost (Q> 0, C> 0).

Detailed evaluation of individual criteria. Decision makers can also use, as aids to their decisions, a less formal method instead of or in addition to

ranking C/E ratios, particularly in situations where using CEA requires considerable subjective estimating of values for the formula. Once criteria have been established, such as by selecting from among the foregoing ones, compliance with each criterion can be assessed and the evaluations combined subjectively, based on each criterion's perceived significance. This type of subjective use of explicit criteria provides a more practical alternative to the more rigorous but also more restrictive quantitative scheme implied by the cost-effectiveness formula. If, however, explicit criteria are used in the decision-making process, the result of such a use of systematically applied professional judgment may well be as sound as if only a CEA had been used.

The most common consideration of industry, when deciding about an R&D investment, is the expectation of recouping the funds and generating profits from selling the product of the research. Some additional factors affecting R&D investment for Class III medical devices—that is, devices that the Food and Drug Administration (FDA) classifies as implantable or life-sustaining—are considered below. Other considerations in industry investment decision making, such as anticipated device efficacy and policies affecting third-party coverage and reimbursement, are discussed in Chapter 4 and Chapter 9.

Several variables may figure significantly in industry R&D decisions, particularly in a decision on whether to invest in MCSS development without NHLBI support. One critical factor is the size of the potential market, which depends on device longevity, potential clinical effectiveness, and third-party coverage and reimbursement. The importance of these factors becomes clear when industry's development of short-term VADs is compared with its current apparent lack of interest in supporting TAH research. Additional crucial factors in health care R&D decisions are the length of time before an investment is recouped and the anticipated difficulty and lack of certainty of obtaining FDA approval. Other typical considerations, particularly with potential Class III devices, include these time and cost factors:

• special testing requirements to determine the product's safety, efficacy, and in vivo durability;

• quality assurance measures that are much more costly for complex medical devices than, for instance, for a chemical compound;

• the length of time before a return on the investment can be realized, because of the delays inherent in the FDA and third-party payer approval processes; and

• the possibility of a substantial exposure to product liability claims, which can be much greater with life-sustaining technologies than with other medical devices, especially because no solution is in sight to what might be called a nationwide product liability crisis.

These considerations are mostly negative ones. They may be at least partially offset by the possibility of revenues from spin-off technologies that emerge from the R&D process, as has occurred with several MCSS developers. Such a potential is, however, difficult to evaluate.

All of these factors are likely to be more important in industry estimates of potential return on investments for long-term TAHs and VADs than are total anticipated R&D costs or the per-unit price of the final device. In the case of TAHs, the companies large enough to make an investment such as would be required to develop a TAH, and carry it through to market, usually have other investment opportunities with far greater certainty of a swift return on R&D costs.

The committee suggests two methods that rely on the formulation and application of explicit criteria to assist NHLBI and other agencies that administer R&D programs in allocating funds among specific programs. The first method simply evaluates R&D options according to explicit criteria. To illustrate the process and form a starting point for NHLBI, the committee identified 18 criteria and ranked them as “ more important,” “important,” or less so; these or similar criteria can be used systematically to provide a rational basis for funding allocation decisions. The second method, useful in some situations, applies cost-effectiveness analysis to R&D program alternatives, thus quantifying one of the most important of the 18 suggested criteria. (Chapter 6 includes an example of a third method, another use of cost-effectiveness analysis, that can aid in decisions within a single R&D program.) Using more explicit criteria, some of them quantitative, can become an increasingly reliable and valuable adjunct to R&D decision making through the exercise of professional judgment.

Hatziandreu, E., J. D. Graham, and M. A. Stoto. 1988. AIDS and biomedical research funding: Comparative analysis. Reviews of Infectious Diseases 10:159-167.

IOM (Institute of Medicine). 1979. DHEW's Research Planning Principles: A Review. Washington, D.C.: National Academy of Sciences.

IOM. 1985. New Vaccine Development: Establishing Priorities. Vol. 1. Diseases of Importance in the United States. Washington, D.C.: National Academy Press.

IOM. 1989. Effectiveness Initiative: Setting Priorities for Clinical Conditions . Washington, D.C.: National Academy Press.

IOM. 1990. National Priorities for the Assessment of Clinical Conditions and Medical Technologies. Washington, D.C.: National Academy Press.

Poirier, V. L. 1990. Presentation at IOM committee public meeting (July 13).

Steinberg, E. P., E. J. Topol, J. W. Sakin, S. N. Kahane, L. J. Appel, N. R. Powe, et al. 1988. Cost and procedure implications of thrombolytic therapy for acute myocardial infarction. Journal of the American College of Cardiology 12:58A-68A.

Thomas, L. 1972. Aspects of Biomedical Science Policy. IOM Occasional Paper. Washington, D.C.: Institute of Medicine.

Weinstein, M. C. 1983. Cost-effective priorities for cancer prevention. Science 221:17-23.