Click for next page ( 46


The National Academies | 500 Fifth St. N.W. | Washington, D.C. 20001
Copyright © National Academy of Sciences. All rights reserved.
Terms of Use and Privacy Statement



Below are the first 10 and last 10 pages of uncorrected machine-read text (when available) of this chapter, followed by the top 30 algorithmically extracted key phrases from the chapter as a whole.
Intended to provide our own search engines and external engines with highly rich, chapter-representative searchable text on the opening pages of each chapter. Because it is UNCORRECTED material, please consider the following text as a useful but insufficient proxy for the authoritative book pages.

Do not use for reproduction, copying, pasting, or reading; exclusively for search engines.

OCR for page 45
? Chapter 4 HEALTH SERVICES RESEARCH IN THE FEDERAL GOVERNMENT The principal focus of the study was the federal government's role as sponsor, producer, and consumer of health services research. Specifi- cally, the committee addressed issues concerning the nature and extent of investments in this area, the manner in which health services research is organized within the federal structure, and the means by which the quality of studies is assured. This chapter presents findings and recommendations on these issues. Federal Involvement in Health Services Research To identify the agencies that might be engaged in health services research, the committee reviewed several published analyses of federal spending for health-related research and development and statistical activities. These reviews revealed that existing reports do not con- sistently and reliably record health services research as defined in Chapter 2. The most inclusive routinely available source of information about health-related research is the annual analysis of the federal budget published by the Office of Management and Budget. According to its analysis of the 1977 federal budget, all executive departments except Housing and Urban Development were engaged in some form of "health research."tl] This category, however, includes several activities and types of research that are not health services research according to the committee's definition, for instance, biomedical research, developmental activities, and routine gathering of statistics. Data published by the National Institutes of Health on federal expendi-- tures for health-related research and development for fiscal year 1975 itemized agencies' activities in biomedical, health services, and other research and development.~2] According to this analysis, only four executive departments were involved in health services research and development: the Department of Health, Education, and Welfare; the Department of Defense; the Energy Research and Development Admini- stration; and the Veterans Administration. Based on the committee's 45

OCR for page 45
46 experience, this inventory was judged to have serious omissions. Within the Department of Health, Education, and Welfare, for example, the report omitted the National Institutes of Health, although the committee was aware of several projects supported by NIH that it considered instances of health services research as research and development.* Therefore, the committee found it necessary to gather information on health services research directly from individual agencies. Fran its contacts with all executive departments and research agencies of the Congress, the committee identified health services research in the following locations: Department of Health, Education, and Welfare Department of Defense Department of State Department of Labor Veterans Administration National Aeronautics and Space Administration Additionally, three of the research arms of Congress--the General Accounting Office, the Congressional Budget Office, and the Office of Technology Assessment--and the Federal Trade Commission have some involvement in this area of research.** Most studies of health services sponsored or conducted by federal agencies are adjuncts to their programmatic missions and constitute only small portions of these missions. The Department of Defense, for example, operates an extensive health services system for active military person- nel and their dependents. In this capacity the Department conducts re- search on the organization, costs, and other features of these services. Similarly, health services research within the Veterans Administration is primarily on the VA hospital system. The Agency for International Develop- ment of the Department of State provides assistance to other nations that includes research and technical assistance for the development of personal health services. The Department of Labor's concerns with labor force participation, collective bargaining, and wage rates encompass workers the health services industry. In the Congress, the General Accounting Office assesses federally funded health programs, and the Congressional Office studies the potential costs of proposed health legislation. *A version of the NIH inventory that is currently being compiled will include the NIH among agencies supporting health services research. **Undoubtedly, other agencies conduct studies from time to time that would be considered health services research. Such studies, however, are usually small-scale and sporadic.

OCR for page 45
47 Nearly all of the agencies and offices of the Department of Health, Education, and Welfare are engaged in some form of health services research. Like activities in other departments, most health services research is mission-oriented and accounts for relatively small portions of agencies' resources. The Health Care Financing Administration and all six agencies of the Public Health Service conduct health services research; each of the offices of planning and evaluation sponsor addi- tional research to inform their policy decisions. (Their organization interrelationships are displayed in Figure 4.) Only the National Center for Health Services Research, the National Center for Health Statistics, and the National Institutes of Health engage in health-related research as a full-time activity, and only the former two concentrate primarily on health services research. The committee found that five federal agencies account for the majority of health services research supported by the federal government: National Center for Health Services Research (NCHSR) National Center for Health Statistics (NCHS) National Institutes of Health (NIH) Alcohol, Drug Abuse, and Mental Health Administration (AD AMHA) Health Care Financing Administration (HCFA) The priorities of these agencies set the agenda for most health services research sponsored by the federal govenment. The National Center for Health Services Research, the only agency of the five with an exclusive mandate to support health services research, sponsors a broad array of research activities both intramurally and extramurally The Center was created in 1968 for that purpose and has no other programmatic mission. It is discussed in detail in Chapter 5. The National Center for Health Statistics is the primary agency for the production of national general purpose health statistics. Findings from its inventories and surveys constitute descriptive health services research; NCHS also conducts special surveys to meet particular research needs such as the national health expenditures survey, which is a joint activity with NCHSR. The mission of the National Institutes of Health has extended beyond the support of biomedical research and development to include a range of activities relating to the widespread application and use of new and available knowledge and techniques to reduce the effects of particular diseases. Though the distinctions are frequently difficult to draw, many of the activities constitute health services research. They are found primarily in comprehensive centers and control programs for cancer, diabetes, arthritis, and cardiovascular and pulmonary diseases, as well as individual demonstration and education projects.

OCR for page 45
Ad: A o H En ~ O (3 En Z En En o U] En Hi o Cal o lo: En ¢ _l - fir Cal IL U) ~ o ._ ._ C) ~ ~--C T 1 C TIC no C, Co o.- s c 4-. - I AS 4 - in s 4- ~ C 4- I '4: i .- CO In so _ I o C C. o ._ I to ~ s In cat 4- ~ c v ~ ~ 4 - ~ >.- `-o CD C1 ~o C~ Ct: .O, ~C I o c ._ ~ o C.~ 0 ^-~ ·- ~ > — C Wc C. W .0.0 0 0 Z ~ ~W aW mo .~._ ,- .'Cc I ~ Cr-~W C C I ~ 1~ ._ C —VC _ ~ V C C C~ ° o ~ ~o —v ~ .Q u, C ~ ~ ~o ~ Co ~ 4 - ~ 4 - .— C . - ~ C I _ 4- .>— ._ _, ~C C c ~ C o o — ·— o ._ ._ ~ C] ._ 4 - ~ ~ C~ ZZ Z c 'c ° _ o W o , 4- .— ~ ~ · ~ C C) ~ C V ~ ~ ._ ~ en C ~ ~ C ~ Z, o ~ ~ ~ ·m ~ ~ ~ . C E ° ° `~o ° c, ° ~ ~ ° ~ ~ ~ ~ C ~ ° —v ~ I ~ ~ ~ ~ ~ ~ C) c ~ V- - · - ~ ·~'C.) · -—V W- - C V C C _ C co ~ C c _ W _ W _ V ,~ W ~ <0 I ·~, ~ · - , ~c ·4., ~ . - o ~ ._ ~ CD CO ~ ~ Z Z Z Z Z 4 - c , ~ — V V _ ~ V t ~ ~ ~ o ~ o c~ a~ c~ ~ I a) . . m m ~n i~ .v I a) c ~ ._ - v ._ a) cn ~ ~ C~ I ~ I ~ o ~ o Q o , C ~ to ~o . . m m V C C ~ .C ~ O Cc ° ~ ~: ~' —0 ~C 0 C~ ~c ~ dO o . - -. - o o co o ~ · - ~ -c 4 - ~ ~ ~ ~ - - ~ ~- <: ~ - - 3 , ~ __ C o o_ ~ ~C ~C °' °' mC, -= o ~ o__ o — o _ .,. ~ 4~ <~: ·—C] c ~ ro ~o ~ o. C Z Z Z <: 48

OCR for page 45
49 Although the Alcohol, Drug Abuse, and Mental Health Administration concentrates on supporting approximately 650 community mental health centers and other service programs throughout the nation, research programs also are sponsored, ranging from physiological and behavioral re- search to health services research. The last includes developmental and evaluation projects, as well as research on the financing, organization, productivity, and need for mental health services and their integration into the general health care sector. The Health Care Financing Administration (HCFA) supports research relating to its responsibilities for Medicare, Medicaid, and professional standards review, their accompanying statistical and monitoring activities, and the eventuality of national health insurance. The mandate of HCFA's Office of Policy Planning and Research is broadly interpreted, and most types of health services research could fall within its purview. Federal Expenditures for Health Services Research Health services research has several different methods of support by the federal government, including intramural activities of federal employees and extramural studies performed by nonfederal persons. Funds for research derive from specific Congressional authorizations, either for particular types of research or for discretionary use by agencies, and from agencies' operating budgets. Support for extramural research is provided through grants and contracts. Grants typically are awarded on the basis of scientific merit and the relevance of research proposals to the funding agency's mission and priorities. Applications for grants usually are initiated by investigators. Contracts are means by which agencies purchase studies from nonfederal persons. Most contracting involves competitive bidding by potential contractors for research tasks conceived and advertised by government agencies. In limited instances, when the task can be performed by only a particular person or institution known to the agency, the competitive process is suspended, and a sole source contract is awarded. The Intergovernmental Personnel Act and service fellows programs provide another mechanism for facilitating research that lies between the tradi- tional intramural and extramural programs. Under these arrangements, nonfederal employees are given the temporary status of federal employees in order to do intramural research. Usually, investigators perform their research at the agency's offices in the Washington, D.C. area. In reviewing these programs, the committee attempted to determine each agency's 1977 intramural and extramural expenditures for health services research. Although the committee endeavored to include only research activities meeting its operational definition, several difficulties were

OCR for page 45
50 encountered. Most important were problems of definition. Records maintained by federal agencies do not reliably and consistently distin- guish funds invested in health services research frog those devoted to other types of research or to routine data collection and reporting for program management. Enumerating funds for health services research was especially difficult in agencies that supported large-scale demonstration and education projects in which most funds were devoted to service activities. Many of the health services research studies sponsored by the National Institutes of Health, for example, are appended to develop- mental projects. In many instances, the costs of evaluating these demonstration projects are quite small; and because they are buried in the total costs of projects, they cannot be estimated precisely. Additionally, demonstration projects raise judgmental questions about whether their total costs should be classified as expenditures for research. Because many developmental efforts officially categorized as demonstration projects are pursued with minimal systematic evaluation, an argument could be made for Omitting them from the enumeration of health services research. On the other hand, since such projects, in principle, are intended to test innovations, their total costs might reasonably be considered research. The committee attempted to segregate the costs of purely developmental activities frog those of related research and evaluation efforts. How- ever, this proved to be an impossible task because of the ways in which the agencies record research budgets. Therefore, in some instances the estimates given below are probably biased upwards. The committee estimates the current federal investment in health services research to be in the neighborhood of $142 million.* As shown in Table 1, expenditures by the Department of Health, Education, and Welfare account for about 85 percent of this total, with no other department or agency contributing more than 8 percent. While these sums are not inconsiderable, they are miniscule in comparison with all spending for health care and account for only a small fraction of the government's total investments in health-related research and statistical activities. As shown in Table 2, the federal government expended less than one dollar for health services research for each $1,000 spent on health care in the United States in 1977 and less than three dollars for each federal outlay of $1,000 for health care. Federal spending for health services research in that year accounted for less than five percent of all outlays for health-related research and statistical activities. *Private foundations contribute another $26.4 million.~3] Data are not available from states and private industry.

OCR for page 45
51 TABLE 1 ESTIMATED EXPENDITURES FOR HEALTH SERVICES RESEARCH BY AGENCY, FISCAL YEAR 1977 Expenditure Percent Agency (in $1,000s) of total Executive Departments (total) (141,118.7) (98.9) Health, Education, and Welfare 121,837.7 85.5 State 10,029.0 7.0 Defense 4,981.0 3.5 Veterans Administration 4,100.0 2.9 National Aeronautics and 100.0 0.1 Space Administration Labor 71.0 0.1 Congressional Agencies (total) (1,277.5) (1.0) General Accounting Office 787.5 0.6 Congressional Budget Office 225.0 0.2 Office of Technology Assessment 265.0 0.2 Federal Trade Commission 175.0 0.1 Total $142,571.2 100.0

OCR for page 45
By hi Ad lo; By H EN U) Cal U] ~ EN ¢ ¢ EN ~ Cal lo: ¢ 1 us EN ~ o C' z a a U) ¢" - o U. EN H z P Z 1— ¢ PA ~ ¢ ¢ C; ¢ U) TIC 1 0 a) SO ~ ED V . - ~ a aJ ~ ~ CO ~ U] En ~ C) a) ~ ~ CO o . - · - En · - ~ o ~o ¢ o cn a · - x - 1 o o c~ ~ · · — o o ~ o ~ ~ o ~ ~ o ~ ~ o ~ ~ v · - ~ ~ · - s~ ~ ~ ~ ~ u' O ~ 1 s~ · o ~ t2 c c) · ~ ~: t v ~ aJ ~ · - C ~ ~ · - a) 0 ~ ~ ~ c' ~ E~ ~ S" S~ ¢ 52 ~C o . - C) a . U) cn oo cn ~ ¢ s" ·,' a, U) oo o a a) ~ u ~ u) o ~ · - v ~ · - o ~ ·e V o cn ~C V a' S~ ce V · - S~ ~n 0` ~ · ~ 00 0 U~ C~ C~ C~ o ·rl C~ a, 1 ·- cn a co a C~ E~ o co ~C ~C

OCR for page 45
53 The majority of federal expenditures for health services research support either intramural or contracted studies. In 1977, about 30 percent of all federal support went to intramural research; approximately 45 percent supported extramural contracted research; and the remaining 25 percent was invested in research grants. As noted earlier, the Department of Health, Education, and Welfare is the principal source of support for health services research. Within the Department, expenditures are concentrated in five agencies (see Table 3) National Center for Health Services Research National Center for Health Statistics National Institutes of Health* Alcohol, Drug Abuse, and Mental Health Administration Health Care Financing Administration Together, these agencies accounted for about 80 percent off all 1977 DREW expenditures for health services research and about 70 percent of all federal expenditures in this area. Emphases of Federally Supported Health Services Research Because most studies of health services research are sponsored by agencies as adjuncts of their operating missions, and because these missions are defined in various ways, the research focuses of federal agencies emphasize different features of related questions. Agencies are variously charged with providing services to particular population groups (e.g., American Indians, the active military, veterans), improving services for particular problems (e.g., mental, dental, specific diseases), and dealing with systemic problems (e.g., financing, planning, manpower development, restraint of trade). *Committee members familiar with the research programs of the NIH noted that officials of same of the institutes estimate NIH's health services research expenditures considerably higher than those shown in Table 1. In the committee's judgment, many of the activities contributing to these higher estimates should not be considered health services research, according to the committee's definition.

OCR for page 45
54 TABLE 3 DEPARTMENT OF HEALTH, EDUCATION, AND WELFARE ESTIMATED EXPENDITURES FOR HEALTH SERVICES RESEARCH BY AGENCY, FISCAL YEAR 1977* . Expenditure Percent Agency (in $1,000s) of Total Office of the Assistant Secretary for Planning and Evaluation/Health Public Health Service (total) Office of Deputy Assistant Secretary for Health Policy, Research, and Statis- tics (total) Office of Health Policy, Research, and Statistics 389.9 National Center for Health Services Research National Center for Health Statistics Health Resources Administration (total) Bureau of Health Manpower Bureau of Health Planning and Resources Development Health Services Administration (total) Bureau of Community Health Services Bureau of Medical Services Indian Health Services National Institutes of Health Center for Disease Control Alcohol, Drug Abuse, and Mental Health Administration 21,161.4 24,039.0 (4,955.2) 4,151.4 803.8 (8,211.9) 5,300.2 1,778.7 1,133.0 19,420.4 870.2 6,935.8 Food and Drug Administration*** 1, 651.2 Health Care Financing Administration 26,832.7 4,870.0 3.9 (90,135.0) (74.1) (48,090.3~** (39.6) 0.3 17.4 19.7 (4.1) 3.4 0.7 (6.7) 4.4 1.5 0.9 15.9 0.7 5.7 22.0 Total $121,837.7 100.0 *Basic data from survey conducted by the Director, Division of Health Budget Analysis, DREW, January 1978, with augmentation and revision based on IOM data. **Includes $2.5 million of Public Health Service evaluation monies. ***Estimate for FY 76.

OCR for page 45
55 As indicated by Figure 5, the division of program emphases within the federal structure creates areas of programmatic overlaps that are reflected in the research interests of various agencies. The Alcohol, Drug Abuse, and Mental Health Administration (ADAMHA), for instance, sponsors federally supported mental health programs. It is concerned with financing, planning, and manpower issues that affect the delivery of mental health services. The Bureau of Health Planning and Resources Development (BHPRD), which is responsible for facilitating comprehen- sive health planning in states and regions, is concerned with all types of services, including mental health services. Finally, the Health Care Financing Administration (HCFA), which manages the federal Medicare and Medicaid programs, seeks ways to contain the costs of mental and other health services to Medicare beneficiaires. Intersecting needs, such as those of ADAMHA, BHPRD, and HCFA in this example, multiply throughout the federal government. A meaningful taxonomy of health services research would classify projects along several dimensions such as those employed in Figure 5. No such taxonomy exists, and the committee's attempt to develop one was thwarted by the paucity of detailed and consistent descriptions of research projects. However, a study of the health services research activities of several D HEW agencies recently undertaken by the Depart- ment provides some insight into how the agencies describe their research focuses. The study revealed that about one-fifth of the agencies'* extramural projects and one-half of their funds were devoted to questions relevant to health insurance, compliance with federal programs, and expenditures for health care (Table 4~. A greater number of projects and slightly over 20 percent of funds focused on quality of care and service delivery questions. Matters pertaining to technology assessment, planning and regulation, health manpower, and health care for the dis- advantaged received less attention, as indicated by both the numbers of studies initiated and the funds devoted to them. As might be expected, the several agencies surveyed classified their projects in categories corresponding to their own principal missions (Table 5~. HCFA, for instance, concentrated 86 percent of its research funds on studies of health insurance and health care expenditures; the Bureau of Health Manpower classified all of its studies in the "health *The study covered all agencies of the Public Health Service (except the National Center for Health Statistics, the Center for Disease Control, the Food and Drug Administration, and the National Institutes of Health) and the Health Care Financing Administration. Intramural projects other than those of the National Center for Health Services Research were excluded, as were projects funded by Public Health Service evaluation monies.~4] Within the Health Resources Administration, nursing and dental health services research activities were omitted.

OCR for page 45
64 Fragmentation of responsibilities also has implications for the quality of research done intramurally and supported by government agencies. The committee found that persons in charge of research in several agencies have limited contact with their counterparts in other agencies and frequently are unaware of research efforts related to their own interests. Hence, opportunities for potentially fruitful collaboration and learning are missed, resulting in substantial variations among agencies in the standards employed in designing and evaluating research projects. The existing organization of research activities is rooted in basic processes of the federal government that militate against coordination. Due to the "from-the-bottom-up" manner in which divisions and depart- ments are constituted, agencies have considerable autonomy and discretion. The Congress mandates agencies' programs and establishes their budgets, frequently earmarking funds for research purposes. Superordinate layers of departments, therefore, have limited control over the program and daily activities of their constituent agencies and no authority over agencies in other divisions and departments of the government. DREW, for instance, has no official involvement in the health services research programs of the Department of Defense or the Veterans Administration; within DREW, the Public Health Service housing NCHSR and NCHS is statutorily and administratively separate from the Health Care Financing Administration which has substantial programs in health services research. This pattern continues through the agency level, where responsibilities for research are divided among divisions and branches. Because health services research activities at each layer of govern- ment usually account for only miniscule portions of its total budget and are peripheral to its principal concerns, these activities receive relatively little attention. As one proceeds upward from the levels where particular projects are conceived or funded, each layer of organization involves fewer people and larger spans of responsibility for greater varieties of problems. Moreover, as needs for and uses of information broaden from concerns with particular programs to attention to agency and departmental policy, decisions affecting research priorities are increasingly colored by conflicting values and other political considerations. Unless systematic mechanisms are established to counteract centrifugal forces that inhere in the organization of the federal government, no coherent research policy or priorities will develop. The committee found few such mechanisms. Although responsibilities for coordination of health services research exist in specific agencies at each layer of government, none devotes sufficient attention to the organizational and substantive problems of health services research. The inability of these agencies to establish priorities and policies stems, in large part, from the disarray of information about the research priorities and emphases of agencies below them. The difficulties encountered by the committee in its attempt to determine the focuses

OCR for page 45
65 and contents of agencies' health services research agendas are indicative of the problems faced by agencies charged with developing priorities and policies. There is, for instance, no routine reporting system that reliably and consistently assembles either descriptions of health services research projects or their results. With notable exceptions,* few agencies routinely produce summaries of their health services research priorities, projects, or findings, and agencies' inventories and records are inconsistent and incomplete. In view of its findings of the widespread involvement in health services research by agencies throughout the federal government, the absence of systematic and effective mechanisms for coordinating activi- ties of departments and agencies, and the consequent problems of fragmentation and omissions in health services research, the committee recommends that administrative procedures be established within _ the federal government to coordinate the setting of departmental and agency health services re- search priorities, agendas, and projects. These procedures should apply to all departments engaged in health services research, and should emphasize the identification of areas of common interest among departments and agencies and, in such instances, facilitate interdepartmental and interagency exchange of information and collaboration. The committee further believes that efforts to coordinate health services research priorities, agendas, and projects should not hamper agencies' abilities to carry out their mandated missions and should encourage experimentation with diverse perspectives and approaches to problems. Therefore, the committee recommends that attempts to coordinate health services research within the federal government should not centra- lize responsibility for the conduct or sponsorship _ . . Of research required for the attainment of specific - and identifiable program or agency objectives. This recommendation has two implications. First, the committee would not endorse a research plan (either government-wide or DHEW-wide) that *The Health Care Financing Administration, The National Center for Health Services Research, and the National Center for Health Statistics.

OCR for page 45
66 would limit the scope or content of agencies' research agendas if they can be demonstrated to be reasonably related to agencies' mandated missions. Second, the committee would not be in favor of a reorgani- zation of health services research that would remove responsibilities for the conduct or sponsorship of programmatic research frog operating agencies. In view of its findings of important matters missing from the research priorities of individual agencies within the Department of Health, Education, and Welfare, created by the close identification of agencies' health services research priorities with their program missions, the committee recommends that agencies be identified to assume responsibilities for implementing studies to bridge the gaps in knowledge. These agencies should periodically review their own research agendas and those of other agencies with common or logically related interests, identify research needs that are not being met, and propose projects that would meet these needs. These findings and plans should be submitted to higher departmental officials who, in turn, should identify agencies and resources to implement them. Quality Controls The quality of research traditionally has been maintained in the scientific community by publication of methods and findings. Completed projects submitted for publication are reviewed by peers to determine whether they satisfy accepted standards of scientific rigor and con- tribute to knowledge. Failing either, the manuscript is not accepted for publication. Dissemination of published research incorporates the mechanisms of review, comment, and debate among peers to correct results when initial reviews are shown to be erroneous or when new knowledge is produced. When the federal government established programs to support scientific research, it adopted peer review as the principal means of assessing potential quality and procedures employed by the National Institutes of Health as its exemplar. Basically, the process entails the review of investigator-initiated research proposals by panels of peers ("study sections") who have contributed to the literature in the fields they review. Applications for support are examined to determine whether methods and subjects of investigation proposed by investigators are likely to contribute significantly to scientific knowledge and whether the investigators are potentially capable of carrying out the projects

OCR for page 45
67 they propose. The latter is assessed by examining the investigators' records of previous performance or, in the case of scientists beginning their careers, by considering their training or recommendations by their supervisors. Proposals disapproved by the panels are not funded by the institutes; those endorsed by panels are assigned priority scores reflecting panel members' judgments of their relative scientific importance. The final step is a substantive review by Institute staff to select from the approved applications those that promise to contribute to the agency's own priorities. Funded projects are subsequently assessed through monitoring of progress and, ultimately, by their contributions to the published literature. The general features of these approaches for assuring the quality of research were adopted by agencies that first offered extramural support for health services research. Currently, however, only a few agencies adhere closely to them. Several circumstances and trends account for this. Above all, the pure form of scientific review has rarely been applied in the field of health services research. The objectives and needs of most sponsoring agencies call for information to be used for various applied purposes as well as to contribute to the accumulation of knowledge relevant to their missions. In consequence, the worthiness of research proposals has been judged in terms of the likelihood that they will provide the information needed by the sponsoring agency, as well as on grounds of scientific merit. As programmatic needs for particular types of information have increased throughout the government, use of the contract mechanism to support ex- tramural research has grown, and intramural activities have enlarged. Both devices provide agencies greater control over the content of research projects but at the expense of opportunities for applying of traditional methods of assuring quality. Contract Research Approximately 45 percent of the federal government's total spending for health services research and about 78 percent of its outlays for extra- mural studies are disbursed through contract mechanisms. Under these procedures, the questions to be addressed and basic designs of research projects are formulated by agency personnel and advertised as requests for proposals. Submissions are usually reviewed either by the government employees who designed the requests or by ad hoc groups of personnel assembled from the agency or other parts of governments In some instances, these groups include nongovernmental persons selected by agency personnel.

OCR for page 45
68 Because the products of contracted research are technically the property of the federal government, agency personnel have at least some control over how and to whom they are disseminated. In some cases, contracts contain provisions that prohibit the contractor from publishing results. The means by which contract proposals are devised and awarded and con- straints on the publication of results combine in many instances to eliminate outside review of the quality of research. Projects are devised and advertised, proposals are reviewed and funded, and results are acquired -- all by the same government personnel. This pattern is particularly common in the large numbers of contracts issued to proprie- tary research firms. As most of these businesses are not generally concerned with the development of a body of knowledge about health services, they rarely publish in journals where their conceptual and methodological approaches and findings could be reviewed by the health services research community. The committee was particularly concerned about the absence of procedures for systematic and open review of relatively large-scale projects. A major virtue of competitive investigator-initiated research coupled with open peer review is its stimulation and assessment of innovative ideas for research within the communities of researchers and health care professionals. This situation does not prevail under contracting mechanisms, for research ideas are generated and evaluated completely within the government. Hence, the conceptual and methodological rigor of requests for proposals are not subject to open assessment, either at the point of their formulation or when proposals are reviewed. Even when nongovernment persons are employed in the review of proposals, their role is often limited primarily to assessing the purely technical and logistical aspects of submissions within the constraints imposed by the conceptual and methodological approaches already fixed by the requests for proposals. These circumstances deter qualified researchers from participating in contract reviews. Having found that substantial portions of federal spending for health services research are disbursed for extramural studies, the majority through contracts, and that most of these disbursements are made without the benefit of systematic and open peer review, the committee recommends that all Executive departments and agencies sponsoring extramural studies in health services research establish peer review by nongovernment personnel of all projects involving appreciable expenditures. These procedures should (1) subject requests for proposals to review prior to their being advertised, (2) facilitate competition for funds among qualified researchers, and (3) review results of projects for their scientific and technical merit.

OCR for page 45
69 Intramural Research Intramural research usually is conducted entirely by government personnel on projects conceived and executed for specific uses by government agencies or officials. Studies carried out by staff of the Federal Trade Commission, for instance, are used by the Commission in rule-making, and those of the General Accounting Office are submitted to Congressional committees. In other cases, government agencies produce information for general dissemination, such as the statistical series on federal expenditures for health services for Medicare beneficiaries published by the Health Care Financing Administration and the various series published by the National Center for Health Statistics. Nearly 30 percent~of all federal expenditures for health services research are for intramural activities. The growth of intramural research raises issues more fundamental than the problems of contract research, although they are similar in some respects. In both cases, the research agenda is set by government agencies and officials, thereby limiting the range of questions that are or could be addressed. Also, both are often conducted in response to specific requests by agencies or officials or as adjuncts to the normal business of government. In these instances, the imposition of peer review on intramural research would inappropriately interfere with wholly internal matters of government.* In the case of intramural research to produce statistical series for general dissemination, the need for prior peer review is partially obviated by the opportunities subsequently afforded the research community to debate publicly the methods and interpretations of published studies. The major issue raised by intramural research is not, therefore, that of peer review. Rather, it has to do with the effects this approach could conceivably have on the quality and content of the entire body of knowledge of health services research. To the extent that intra- mural research is used as a substitute for extramural research, the types of problems addressed and approaches used are determined increas- ingly by federal personnel, relegating the research community's roles to those of occasional advisors and critics of published results. The long term consequences of this strategy would have serious dele- terious effects on the types of research done and on its quality. As noted earlier, the committee believes that there must be opportunities for replication of studies in health services research to guard against *However, in instances where large-scale studies are undertaken intra- murally, the committee encourages the use of advisory groups to assist in their designs, to oversee their implementation, and to review their findings and interpretations.

OCR for page 45
70 basing health care policy decisions on only a few studies of complex questions. Given the pressures within the federal government toward standardization of definitions and methods, a totally intramural strategy for health services research could greatly impair the process by which previous research results are subjected to revision by the appearance of contradictory findings based on different conceptual and methodological approaches. In the same vein, an exclusively intramural strategy would, in effect, create a government monopoly over data that would contribute to the lack of opportunities to challenge results. Ultimately, the consequences of this strategy would be seen in the migration of qualified and interested researchers to other fields of inquiry and the destruction of the infrastructure of the field of health services research. Without this community of researchers and their work to draw upon, the quality of federally supported intramural research would surely decline. The question of where to draw the boundaries between intramural and extramural efforts in health services research might be addressed in terms of the contents of research questions and the government's needs for information.~6] The intersection of these features in Figure 6 creates four types of situations and identifies funding strategies most suited to each. Situations A and B include studies of questions for which relatively well-established and codified conceptual and methodological solutions exist, for instance, the enumeration of physician visits using house- hold surveys. Situations C and D, by contrast, involve problems for which there are no standard solutions, as for instance questions about the economic value of life. The columns distinguish situations in which needs for information are either highly targeted or routine (A & C) from those in which needs are less well-defined or predictable (B & D). The cells of Figure 6 indicate the types of funding strategies that the committee believes best fit these situations. Intramural research is best suited to deal with the problems for which standard solutions exist and for which there are high specific or routine needs for infor- mation. Most studies performed by the National Center for Health Statistics are in this category. When needs for such information are less specific or routine, either the contract mechanism or the intra- mural strategy is suitable. Here, the contract might be used as an adjunct to or extension of intramural activities. Problems for which no routine solution is available call for competitive proposals, either for contracts in circumstances where needs for information are specific or routine or for grants in other situations. Finding that federal agencies are increasingly relying upon intramural research and research funded by contract, and being concerned over the long-term consequences of these funding strategies for the types of

OCR for page 45
71 FIGURE 6 RESEARCH FUNDING STRATEGY Needs for Information Targeted or Routine Well- established State-of- the-art Not well- established Not Targeted or Routine A B Intramural Contract or I Intramural ... C D Contract Grant .

OCR for page 45
72 research that will be done and for the quality of health services research, the committee recommends that the federal government adopt a policy regarding health services research to assure that a signifi cant portion of all monies invested in this area go to support investigator-initiated extramural research. - Intramural research should not be viewed as a substitute for extramural research, nor contracted research as a substitute for grant-supported investigator-initiated research. Rather, a strategy of funding should be developed that identifies the strengths and problems associated with each and achieves a balance among them.

OCR for page 45
73 REFERENCES 1 2 3 4 5 6 Office of Management and Budget, "Special Analysis L: Analysis. Budget of the United States Government, 1979 (January 1978), Healthy Special U.S. Department of Health, Education, and Welfare, National Institutes of Health, Office of the Director for Program Planning and Evaluation Dollars for Health Research and Development: 1968-1975, DREW Pub. . no. (NIH) 77-1185 (June 1977): John Craig, Mega McDonald, and Betty Dooley, "Private Foundations' Health Expenditures: A Survey Analysis," Health Policy Research Group, George- town School of Medicine, May, 1978. Report by the Director, Division of Budget Analysis, DREW, January 1978. U.S. Congress, Senate, Committee on Appropriations, Departments of Labor and Health, Education, and Welfare and Related Agencies - Appropriations Bill, 1978, Report no. 95-283, 95th Cong., 1st sees., 1977, p. 81. Gerald Gordon, Ann E. MacEachron, and G. Lawrence Fisher, "A Contin- gency Model for the Design of Problem-Solving Research Programs: A Perspective on Diffusion Research," Milbank Memorial Fund Quarterly/ Health and Society 52 (Spring 1974~: 185-220.

OCR for page 45