|
||||||||||||||||
|
||||||||||||||||
|
|
|||||||||||||||
|
| ||||||||||||||||||||||||||||||||
|
||||||||||||||||||||||||||||||||
Below are the first 10 and last 10 pages of uncorrected machine-read text (when available) of this chapter, followed by the top 30 algorithmically extracted key phrases from the chapter as a whole.
Intended to provide our own search engines and external engines with highly rich, chapter-representative searchable text on the opening pages of each chapter.
Because it is UNCORRECTED material, please consider the following text as a useful but insufficient proxy for the authoritative book pages.
Do not use for reproduction, copying, pasting, or reading; exclusively for search engines.
OCR for page 45
?
Chapter 4
HEALTH SERVICES RESEARCH IN THE FEDERAL GOVERNMENT
The principal focus of the study was the federal government's role as
sponsor, producer, and consumer of health services research. Specifi-
cally, the committee addressed issues concerning the nature and extent
of investments in this area, the manner in which health services
research is organized within the federal structure, and the means by
which the quality of studies is assured. This chapter presents findings
and recommendations on these issues.
Federal Involvement in Health Services Research
To identify the agencies that might be engaged in health services
research, the committee reviewed several published analyses of federal
spending for health-related research and development and statistical
activities. These reviews revealed that existing reports do not con-
sistently and reliably record health services research as defined in
Chapter 2.
The most inclusive routinely available source of information about
health-related research is the annual analysis of the federal budget
published by the Office of Management and Budget. According to its
analysis of the 1977 federal budget, all executive departments except
Housing and Urban Development were engaged in some form of "health
research."tl] This category, however, includes several activities and
types of research that are not health services research according to the
committee's definition, for instance, biomedical research, developmental
activities, and routine gathering of statistics.
Data published by the National Institutes of Health on federal expendi--
tures for health-related research and development for fiscal year 1975
itemized agencies' activities in biomedical, health services, and other
research and development.~2] According to this analysis, only four
executive departments were involved in health services research and
development: the Department of Health, Education, and Welfare; the
Department of Defense; the Energy Research and Development Admini-
stration; and the Veterans Administration. Based on the committee's
45
OCR for page 46
46
experience, this inventory was judged to have serious omissions. Within
the Department of Health, Education, and Welfare, for example, the report
omitted the National Institutes of Health, although the committee was
aware of several projects supported by NIH that it considered instances
of health services research as research and development.* Therefore, the
committee found it necessary to gather information on health services
research directly from individual agencies.
Fran its contacts with all executive departments and research agencies
of the Congress, the committee identified health services research in
the following locations:
Department of Health, Education, and Welfare
Department of Defense
Department of State
Department of Labor
Veterans Administration
National Aeronautics and Space Administration
Additionally, three of the research arms of Congress--the General
Accounting Office, the Congressional Budget Office, and the Office of
Technology Assessment--and the Federal Trade Commission have some
involvement in this area of research.**
Most studies of health services sponsored or conducted by federal agencies
are adjuncts to their programmatic missions and constitute only small
portions of these missions. The Department of Defense, for example,
operates an extensive health services system for active military person-
nel and their dependents. In this capacity the Department conducts re-
search on the organization, costs, and other features of these services.
Similarly, health services research within the Veterans Administration is
primarily on the VA hospital system. The Agency for International Develop-
ment of the Department of State provides assistance to other nations that
includes research and technical assistance for the development of personal
health services. The Department of Labor's concerns with labor force
participation, collective bargaining, and wage rates encompass workers
the health services industry. In the Congress, the General Accounting
Office assesses federally funded health programs, and the Congressional
Office studies the potential costs of proposed health legislation.
*A version of the NIH inventory that is currently being compiled will
include the NIH among agencies supporting health services research.
**Undoubtedly, other agencies conduct studies from time to time that
would be considered health services research. Such studies, however,
are usually small-scale and sporadic.
OCR for page 47
47
Nearly all of the agencies and offices of the Department of Health,
Education, and Welfare are engaged in some form of health services
research. Like activities in other departments, most health services
research is mission-oriented and accounts for relatively small portions
of agencies' resources. The Health Care Financing Administration and
all six agencies of the Public Health Service conduct health services
research; each of the offices of planning and evaluation sponsor addi-
tional research to inform their policy decisions. (Their organization
interrelationships are displayed in Figure 4.) Only the National Center
for Health Services Research, the National Center for Health Statistics,
and the National Institutes of Health engage in health-related research
as a full-time activity, and only the former two concentrate primarily
on health services research.
The committee found that five federal agencies account for the majority
of health services research supported by the federal government:
National Center for Health Services Research (NCHSR)
National Center for Health Statistics (NCHS)
National Institutes of Health (NIH)
Alcohol, Drug Abuse, and Mental Health Administration (AD AMHA)
Health Care Financing Administration (HCFA)
The priorities of these agencies set the agenda for most health services
research sponsored by the federal govenment.
The National Center for Health Services Research, the only agency of
the five with an exclusive mandate to support health services research,
sponsors a broad array of research activities both intramurally and
extramurally The Center was created in 1968 for that purpose and has
no other programmatic mission. It is discussed in detail in Chapter 5.
The National Center for Health Statistics is the primary agency for
the production of national general purpose health statistics. Findings
from its inventories and surveys constitute descriptive health services
research; NCHS also conducts special surveys to meet particular research
needs such as the national health expenditures survey, which is a joint
activity with NCHSR.
The mission of the National Institutes of Health has extended beyond the
support of biomedical research and development to include a range of
activities relating to the widespread application and use of new and
available knowledge and techniques to reduce the effects of particular
diseases. Though the distinctions are frequently difficult to draw,
many of the activities constitute health services research. They are
found primarily in comprehensive centers and control programs for
cancer, diabetes, arthritis, and cardiovascular and pulmonary diseases,
as well as individual demonstration and education projects.
OCR for page 48
Ad:
A
o
H
En
~ O
(3 En
Z
En
En
o
U]
En
Hi
o
Cal
o
lo:
En
¢
_l
-
fir
Cal
IL
U)
~ o
._ ._
C) ~
~--C
T
1
C
TIC
no
C, Co
o.-
s c
4-. -
I AS
4 -
in s
4- ~
C
4- I
'4: i
.-
CO
In
so
_
I
o
C
C.
o
._
I
to
~ s
In cat
4- ~
c v
~ ~ 4 -
~ >.-
`-o CD
C1 ~o C~
Ct:
.O,
~C
I
o
c
._
~ o C.~
0 ^-~
·- ~ >
— C Wc C.
W .0.0 0
0 Z ~ ~W
aW
mo
.~._
,- .'Cc
I ~
Cr-~W
C C
I
~ 1~
._
C —VC
_ ~ V
C C C~
° o ~ ~o
—v ~ .Q u,
C ~ ~ ~o ~
Co ~ 4 - ~ 4 -
.—
C . - ~ C
I _ 4- .>—
._ _,
~C C c ~ C
o o — ·— o
._ ._ ~ C] ._
4 -
~ ~ C~
ZZ Z
c
'c
° _ o W o
, 4- .— ~ ~ · ~ C
C) ~ C V ~ ~ ._ ~ en
C ~ ~ C ~ Z, o ~
~ ~ ·m ~ ~ ~ . C
E ° ° `~o ° c, ° ~ ~ °
~ ~ ~ ~ C ~ ° —v ~
I ~ ~ ~ ~ ~ ~ C) c ~
V- - · - ~ ·~'C.) · -—V W- -
C V C C _ C co ~ C
c _ W _ W _ V ,~ W ~ <0
I ·~, ~ · - , ~c ·4., ~ . - o ~ ._
~ CD CO ~ ~
Z Z Z Z Z
4 -
c
, ~ —
V V
_ ~ V
t ~ ~ ~
o ~ o
c~ a~ c~
~ I a)
. .
m m
~n i~
.v I
a) c
~ ._
-
v
._
a)
cn
~ ~ C~
I ~ I
~ o ~
o Q o
, C ~
to ~o
. .
m m
V C
C ~
.C ~ O
Cc ° ~
~: ~'
—0
~C
0 C~
~c ~
dO
o
. -
-. - o o co o
~ · - ~ -c 4 - ~ ~
~ ~ - - ~ ~- <: ~ - - 3
, ~ __ C o o_ ~
~C ~C °' °' mC,
-= o ~ o__ o —
o _ .,. ~ 4~ <~: ·—C]
c ~ ro ~o ~
o. C Z Z Z
<:
48
OCR for page 49
49
Although the Alcohol, Drug Abuse, and Mental Health Administration
concentrates on supporting approximately 650 community mental health
centers and other service programs throughout the nation, research
programs also are sponsored, ranging from physiological and behavioral re-
search to health services research. The last includes developmental and
evaluation projects, as well as research on the financing, organization,
productivity, and need for mental health services and their integration
into the general health care sector.
The Health Care Financing Administration (HCFA) supports research
relating to its responsibilities for Medicare, Medicaid, and professional
standards review, their accompanying statistical and monitoring activities,
and the eventuality of national health insurance. The mandate of HCFA's
Office of Policy Planning and Research is broadly interpreted, and most
types of health services research could fall within its purview.
Federal Expenditures for Health Services Research
Health services research has several different methods of support by the
federal government, including intramural activities of federal employees
and extramural studies performed by nonfederal persons. Funds for
research derive from specific Congressional authorizations, either for
particular types of research or for discretionary use by agencies, and
from agencies' operating budgets. Support for extramural research is
provided through grants and contracts. Grants typically are awarded on
the basis of scientific merit and the relevance of research proposals
to the funding agency's mission and priorities. Applications for grants
usually are initiated by investigators. Contracts are means by which
agencies purchase studies from nonfederal persons. Most contracting
involves competitive bidding by potential contractors for research tasks
conceived and advertised by government agencies. In limited instances,
when the task can be performed by only a particular person or institution
known to the agency, the competitive process is suspended, and a sole
source contract is awarded.
The Intergovernmental Personnel Act and service fellows programs provide
another mechanism for facilitating research that lies between the tradi-
tional intramural and extramural programs. Under these arrangements,
nonfederal employees are given the temporary status of federal employees
in order to do intramural research. Usually, investigators perform their
research at the agency's offices in the Washington, D.C. area.
In reviewing these programs, the committee attempted to determine each
agency's 1977 intramural and extramural expenditures for health services
research. Although the committee endeavored to include only research
activities meeting its operational definition, several difficulties were
OCR for page 50
50
encountered. Most important were problems of definition. Records
maintained by federal agencies do not reliably and consistently distin-
guish funds invested in health services research frog those devoted to
other types of research or to routine data collection and reporting for
program management. Enumerating funds for health services research was
especially difficult in agencies that supported large-scale demonstration
and education projects in which most funds were devoted to service
activities. Many of the health services research studies sponsored by
the National Institutes of Health, for example, are appended to develop-
mental projects. In many instances, the costs of evaluating these
demonstration projects are quite small; and because they are buried in the
total costs of projects, they cannot be estimated precisely. Additionally,
demonstration projects raise judgmental questions about whether their total
costs should be classified as expenditures for research. Because many
developmental efforts officially categorized as demonstration projects are
pursued with minimal systematic evaluation, an argument could be made for
Omitting them from the enumeration of health services research. On the
other hand, since such projects, in principle, are intended to test
innovations, their total costs might reasonably be considered research.
The committee attempted to segregate the costs of purely developmental
activities frog those of related research and evaluation efforts. How-
ever, this proved to be an impossible task because of the ways in which
the agencies record research budgets. Therefore, in some instances the
estimates given below are probably biased upwards.
The committee estimates the current federal investment in health services
research to be in the neighborhood of $142 million.* As shown in Table 1,
expenditures by the Department of Health, Education, and Welfare account
for about 85 percent of this total, with no other department or agency
contributing more than 8 percent.
While these sums are not inconsiderable, they are miniscule in comparison
with all spending for health care and account for only a small fraction
of the government's total investments in health-related research and
statistical activities. As shown in Table 2, the federal government
expended less than one dollar for health services research for each
$1,000 spent on health care in the United States in 1977 and less than
three dollars for each federal outlay of $1,000 for health care. Federal
spending for health services research in that year accounted for less
than five percent of all outlays for health-related research and statistical
activities.
*Private foundations contribute another $26.4 million.~3] Data are not
available from states and private industry.
OCR for page 51
51
TABLE 1
ESTIMATED EXPENDITURES FOR HEALTH SERVICES RESEARCH
BY AGENCY, FISCAL YEAR 1977
Expenditure Percent
Agency (in $1,000s) of total
Executive Departments (total)
(141,118.7) (98.9)
Health, Education, and Welfare 121,837.7 85.5
State 10,029.0 7.0
Defense 4,981.0 3.5
Veterans Administration 4,100.0 2.9
National Aeronautics and 100.0 0.1
Space Administration
Labor 71.0 0.1
Congressional Agencies (total) (1,277.5) (1.0)
General Accounting Office 787.5 0.6
Congressional Budget Office 225.0 0.2
Office of Technology Assessment 265.0 0.2
Federal Trade Commission 175.0 0.1
Total $142,571.2 100.0
OCR for page 52
By
hi
Ad
lo;
By
H EN
U)
Cal
U]
~ EN
¢ ¢
EN ~
Cal
lo:
¢ 1
us EN
~
o
C'
z
a
a
U)
¢"
- o
U.
EN
H
z
P
Z 1—
¢
PA ~
¢ ¢
C;
¢
U)
TIC
1
0 a)
SO ~
ED V
. - ~ a
aJ ~ ~
CO ~ U]
En ~
C)
a) ~ ~
CO
o
. -
· -
En
· - ~
o
~o
¢ o
cn
a
· -
x
- 1
o
o c~ ~
· · —
o o ~
o ~ ~
o ~ ~
o ~ ~
o ~ ~
v
· -
~ ~ · -
s~ ~ ~
~ ~ u'
O ~ 1
s~ · o ~ t2
c
c) · ~ ~:
t v ~
aJ ~ · -
C ~ ~ · -
a) 0 ~ ~ ~ c'
~ E~ ~ S" S~
¢
52
~C
o
. -
C)
a
. U)
cn oo
cn ~
¢ s"
·,'
a,
U)
oo
o
a
a)
~ u
~ u)
o ~
· -
v ~
· -
o ~
·e
V
o
cn
~C
V
a'
S~
ce
V
· -
S~
~n
0` ~
· ~
00 0
U~
C~
C~
C~
o
·rl
C~
a,
1
·-
cn
a
co
a
C~
E~
o
co
~C
~C
OCR for page 53
53
The majority of federal expenditures for health services research support
either intramural or contracted studies. In 1977, about 30 percent of
all federal support went to intramural research; approximately 45 percent
supported extramural contracted research; and the remaining 25 percent
was invested in research grants.
As noted earlier, the Department of Health, Education, and Welfare is the
principal source of support for health services research. Within the
Department, expenditures are concentrated in five agencies (see Table 3)
National Center for Health Services Research
National Center for Health Statistics
National Institutes of Health*
Alcohol, Drug Abuse, and Mental Health Administration
Health Care Financing Administration
Together, these agencies accounted for about 80 percent off all 1977 DREW
expenditures for health services research and about 70 percent of all
federal expenditures in this area.
Emphases of Federally Supported Health Services Research
Because most studies of health services research are sponsored by
agencies as adjuncts of their operating missions, and because these
missions are defined in various ways, the research focuses of federal
agencies emphasize different features of related questions. Agencies
are variously charged with providing services to particular population
groups (e.g., American Indians, the active military, veterans), improving
services for particular problems (e.g., mental, dental, specific diseases),
and dealing with systemic problems (e.g., financing, planning, manpower
development, restraint of trade).
*Committee members familiar with the research programs of the NIH noted
that officials of same of the institutes estimate NIH's health services
research expenditures considerably higher than those shown in Table 1.
In the committee's judgment, many of the activities contributing to
these higher estimates should not be considered health services research,
according to the committee's definition.
OCR for page 54
54
TABLE 3
DEPARTMENT OF HEALTH, EDUCATION, AND WELFARE ESTIMATED EXPENDITURES
FOR HEALTH SERVICES RESEARCH BY AGENCY, FISCAL YEAR 1977*
.
Expenditure Percent
Agency (in $1,000s) of Total
Office of the Assistant Secretary for Planning
and Evaluation/Health
Public Health Service (total)
Office of Deputy Assistant Secretary for
Health Policy, Research, and Statis-
tics (total)
Office of Health Policy, Research,
and Statistics 389.9
National Center for Health Services
Research
National Center for Health Statistics
Health Resources Administration (total)
Bureau of Health Manpower
Bureau of Health Planning and
Resources Development
Health Services Administration (total)
Bureau of Community Health Services
Bureau of Medical Services
Indian Health Services
National Institutes of Health
Center for Disease Control
Alcohol, Drug Abuse, and Mental Health
Administration
21,161.4
24,039.0
(4,955.2)
4,151.4
803.8
(8,211.9)
5,300.2
1,778.7
1,133.0
19,420.4
870.2
6,935.8
Food and Drug Administration*** 1, 651.2
Health Care Financing Administration 26,832.7
4,870.0 3.9
(90,135.0) (74.1)
(48,090.3~** (39.6)
0.3
17.4
19.7
(4.1)
3.4
0.7
(6.7)
4.4
1.5
0.9
15.9
0.7
5.7
22.0
Total $121,837.7 100.0
*Basic data from survey conducted by the Director, Division of Health
Budget Analysis, DREW, January 1978, with augmentation and revision
based on IOM data.
**Includes $2.5 million of Public Health Service evaluation monies.
***Estimate for FY 76.
OCR for page 55
55
As indicated by Figure 5, the division of program emphases within the
federal structure creates areas of programmatic overlaps that are
reflected in the research interests of various agencies. The Alcohol,
Drug Abuse, and Mental Health Administration (ADAMHA), for instance,
sponsors federally supported mental health programs. It is concerned
with financing, planning, and manpower issues that affect the delivery
of mental health services. The Bureau of Health Planning and Resources
Development (BHPRD), which is responsible for facilitating comprehen-
sive health planning in states and regions, is concerned with all types
of services, including mental health services. Finally, the Health Care
Financing Administration (HCFA), which manages the federal Medicare and
Medicaid programs, seeks ways to contain the costs of mental and other
health services to Medicare beneficiaires. Intersecting needs, such as
those of ADAMHA, BHPRD, and HCFA in this example, multiply throughout
the federal government.
A meaningful taxonomy of health services research would classify
projects along several dimensions such as those employed in Figure 5.
No such taxonomy exists, and the committee's attempt to develop one
was thwarted by the paucity of detailed and consistent descriptions
of research projects. However, a study of the health services research
activities of several D HEW agencies recently undertaken by the Depart-
ment provides some insight into how the agencies describe their research
focuses.
The study revealed that about one-fifth of the agencies'* extramural
projects and one-half of their funds were devoted to questions relevant
to health insurance, compliance with federal programs, and expenditures
for health care (Table 4~. A greater number of projects and slightly
over 20 percent of funds focused on quality of care and service
delivery questions. Matters pertaining to technology assessment,
planning and regulation, health manpower, and health care for the dis-
advantaged received less attention, as indicated by both the numbers
of studies initiated and the funds devoted to them.
As might be expected, the several agencies surveyed classified their
projects in categories corresponding to their own principal missions
(Table 5~. HCFA, for instance, concentrated 86 percent of its research
funds on studies of health insurance and health care expenditures; the
Bureau of Health Manpower classified all of its studies in the "health
*The study covered all agencies of the Public Health Service (except
the National Center for Health Statistics, the Center for Disease Control,
the Food and Drug Administration, and the National Institutes of Health)
and the Health Care Financing Administration. Intramural projects other
than those of the National Center for Health Services Research were
excluded, as were projects funded by Public Health Service evaluation
monies.~4] Within the Health Resources Administration, nursing and
dental health services research activities were omitted.
OCR for page 64
64
Fragmentation of responsibilities also has implications for the quality
of research done intramurally and supported by government agencies. The
committee found that persons in charge of research in several agencies
have limited contact with their counterparts in other agencies and
frequently are unaware of research efforts related to their own interests.
Hence, opportunities for potentially fruitful collaboration and learning
are missed, resulting in substantial variations among agencies in the
standards employed in designing and evaluating research projects.
The existing organization of research activities is rooted in basic
processes of the federal government that militate against coordination.
Due to the "from-the-bottom-up" manner in which divisions and depart-
ments are constituted, agencies have considerable autonomy and discretion.
The Congress mandates agencies' programs and establishes their budgets,
frequently earmarking funds for research purposes. Superordinate
layers of departments, therefore, have limited control over the program
and daily activities of their constituent agencies and no authority
over agencies in other divisions and departments of the government.
DREW, for instance, has no official involvement in the health services
research programs of the Department of Defense or the Veterans
Administration; within DREW, the Public Health Service housing NCHSR
and NCHS is statutorily and administratively separate from the Health
Care Financing Administration which has substantial programs in health
services research. This pattern continues through the agency level,
where responsibilities for research are divided among divisions and
branches.
Because health services research activities at each layer of govern-
ment usually account for only miniscule portions of its total budget
and are peripheral to its principal concerns, these activities receive
relatively little attention. As one proceeds upward from the levels
where particular projects are conceived or funded, each layer of
organization involves fewer people and larger spans of responsibility
for greater varieties of problems. Moreover, as needs for and uses of
information broaden from concerns with particular programs to attention
to agency and departmental policy, decisions affecting research
priorities are increasingly colored by conflicting values and other
political considerations.
Unless systematic mechanisms are established to counteract centrifugal
forces that inhere in the organization of the federal government, no
coherent research policy or priorities will develop. The committee
found few such mechanisms. Although responsibilities for coordination
of health services research exist in specific agencies at each layer
of government, none devotes sufficient attention to the organizational
and substantive problems of health services research.
The inability of these agencies to establish priorities and policies
stems, in large part, from the disarray of information about the research
priorities and emphases of agencies below them. The difficulties
encountered by the committee in its attempt to determine the focuses
OCR for page 65
65
and contents of agencies' health services research agendas are indicative
of the problems faced by agencies charged with developing priorities
and policies. There is, for instance, no routine reporting system that
reliably and consistently assembles either descriptions of health
services research projects or their results. With notable exceptions,*
few agencies routinely produce summaries of their health services
research priorities, projects, or findings, and agencies' inventories
and records are inconsistent and incomplete.
In view of its findings of the widespread involvement in health
services research by agencies throughout the federal government, the
absence of systematic and effective mechanisms for coordinating activi-
ties of departments and agencies, and the consequent problems of
fragmentation and omissions in health services research, the committee
recommends that
administrative procedures be established within
_
the federal government to coordinate the setting
of departmental and agency health services re-
search priorities, agendas, and projects.
These procedures should apply to all departments engaged in health
services research, and should emphasize the identification of areas of
common interest among departments and agencies and, in such instances,
facilitate interdepartmental and interagency exchange of information
and collaboration.
The committee further believes that efforts to coordinate health services
research priorities, agendas, and projects should not hamper agencies'
abilities to carry out their mandated missions and should encourage
experimentation with diverse perspectives and approaches to problems.
Therefore, the committee recommends that
attempts to coordinate health services research
within the federal government should not centra-
lize responsibility for the conduct or sponsorship
_ . .
Of research required for the attainment of specific
-
and identifiable program or agency objectives.
This recommendation has two implications. First, the committee would
not endorse a research plan (either government-wide or DHEW-wide) that
*The Health Care Financing Administration, The National Center for Health
Services Research, and the National Center for Health Statistics.
OCR for page 66
66
would limit the scope or content of agencies' research agendas if they
can be demonstrated to be reasonably related to agencies' mandated
missions. Second, the committee would not be in favor of a reorgani-
zation of health services research that would remove responsibilities
for the conduct or sponsorship of programmatic research frog operating
agencies.
In view of its findings of important matters missing from the research
priorities of individual agencies within the Department of Health,
Education, and Welfare, created by the close identification of agencies'
health services research priorities with their program missions, the
committee recommends that
agencies be identified to assume responsibilities
for implementing studies to bridge the gaps in
knowledge.
These agencies should periodically review their own research agendas
and those of other agencies with common or logically related interests,
identify research needs that are not being met, and propose projects that
would meet these needs. These findings and plans should be submitted
to higher departmental officials who, in turn, should identify agencies
and resources to implement them.
Quality Controls
The quality of research traditionally has been maintained in the
scientific community by publication of methods and findings. Completed
projects submitted for publication are reviewed by peers to determine
whether they satisfy accepted standards of scientific rigor and con-
tribute to knowledge. Failing either, the manuscript is not accepted
for publication. Dissemination of published research incorporates the
mechanisms of review, comment, and debate among peers to correct results
when initial reviews are shown to be erroneous or when new knowledge is
produced.
When the federal government established programs to support scientific
research, it adopted peer review as the principal means of assessing
potential quality and procedures employed by the National Institutes of
Health as its exemplar. Basically, the process entails the review of
investigator-initiated research proposals by panels of peers ("study
sections") who have contributed to the literature in the fields they
review. Applications for support are examined to determine whether
methods and subjects of investigation proposed by investigators are
likely to contribute significantly to scientific knowledge and whether
the investigators are potentially capable of carrying out the projects
OCR for page 67
67
they propose. The latter is assessed by examining the investigators'
records of previous performance or, in the case of scientists beginning
their careers, by considering their training or recommendations by
their supervisors. Proposals disapproved by the panels are not funded
by the institutes; those endorsed by panels are assigned priority scores
reflecting panel members' judgments of their relative scientific
importance. The final step is a substantive review by Institute staff
to select from the approved applications those that promise to contribute
to the agency's own priorities. Funded projects are subsequently assessed
through monitoring of progress and, ultimately, by their contributions
to the published literature.
The general features of these approaches for assuring the quality of
research were adopted by agencies that first offered extramural support
for health services research. Currently, however, only a few agencies
adhere closely to them.
Several circumstances and trends account for this. Above all, the pure
form of scientific review has rarely been applied in the field of health
services research. The objectives and needs of most sponsoring agencies
call for information to be used for various applied purposes as well as
to contribute to the accumulation of knowledge relevant to their
missions. In consequence, the worthiness of research proposals has been
judged in terms of the likelihood that they will provide the information
needed by the sponsoring agency, as well as on grounds of scientific
merit.
As programmatic needs for particular types of information have increased
throughout the government, use of the contract mechanism to support ex-
tramural research has grown, and intramural activities have enlarged.
Both devices provide agencies greater control over the content of research
projects but at the expense of opportunities for applying of traditional
methods of assuring quality.
Contract Research
Approximately 45 percent of the federal government's total spending for
health services research and about 78 percent of its outlays for extra-
mural studies are disbursed through contract mechanisms. Under these
procedures, the questions to be addressed and basic designs of research
projects are formulated by agency personnel and advertised as requests
for proposals. Submissions are usually reviewed either by the government
employees who designed the requests or by ad hoc groups of personnel
assembled from the agency or other parts of governments In some
instances, these groups include nongovernmental persons selected by agency
personnel.
OCR for page 68
68
Because the products of contracted research are technically the property
of the federal government, agency personnel have at least some control
over how and to whom they are disseminated. In some cases, contracts
contain provisions that prohibit the contractor from publishing results.
The means by which contract proposals are devised and awarded and con-
straints on the publication of results combine in many instances to
eliminate outside review of the quality of research. Projects are
devised and advertised, proposals are reviewed and funded, and results
are acquired -- all by the same government personnel. This pattern is
particularly common in the large numbers of contracts issued to proprie-
tary research firms. As most of these businesses are not generally
concerned with the development of a body of knowledge about health
services, they rarely publish in journals where their conceptual and
methodological approaches and findings could be reviewed by the health
services research community.
The committee was particularly concerned about the absence of procedures
for systematic and open review of relatively large-scale projects. A
major virtue of competitive investigator-initiated research coupled with
open peer review is its stimulation and assessment of innovative ideas
for research within the communities of researchers and health care
professionals. This situation does not prevail under contracting
mechanisms, for research ideas are generated and evaluated completely
within the government. Hence, the conceptual and methodological rigor of
requests for proposals are not subject to open assessment, either at the
point of their formulation or when proposals are reviewed. Even when
nongovernment persons are employed in the review of proposals, their
role is often limited primarily to assessing the purely technical and
logistical aspects of submissions within the constraints imposed by the
conceptual and methodological approaches already fixed by the requests
for proposals. These circumstances deter qualified researchers from
participating in contract reviews.
Having found that substantial portions of federal spending for health
services research are disbursed for extramural studies, the majority
through contracts, and that most of these disbursements are made without
the benefit of systematic and open peer review, the committee recommends
that
all Executive departments and agencies sponsoring
extramural studies in health services research
establish peer review by nongovernment personnel
of all projects involving appreciable expenditures.
These procedures should (1) subject requests for proposals to review
prior to their being advertised, (2) facilitate competition for funds
among qualified researchers, and (3) review results of projects for
their scientific and technical merit.
OCR for page 69
69
Intramural Research
Intramural research usually is conducted entirely by government
personnel on projects conceived and executed for specific uses by
government agencies or officials. Studies carried out by staff of the
Federal Trade Commission, for instance, are used by the Commission in
rule-making, and those of the General Accounting Office are submitted to
Congressional committees. In other cases, government agencies produce
information for general dissemination, such as the statistical series
on federal expenditures for health services for Medicare beneficiaries
published by the Health Care Financing Administration and the various
series published by the National Center for Health Statistics. Nearly
30 percent~of all federal expenditures for health services research are
for intramural activities.
The growth of intramural research raises issues more fundamental than
the problems of contract research, although they are similar in some
respects. In both cases, the research agenda is set by government
agencies and officials, thereby limiting the range of questions that
are or could be addressed. Also, both are often conducted in response
to specific requests by agencies or officials or as adjuncts to the
normal business of government. In these instances, the imposition of
peer review on intramural research would inappropriately interfere
with wholly internal matters of government.* In the case of intramural
research to produce statistical series for general dissemination, the
need for prior peer review is partially obviated by the opportunities
subsequently afforded the research community to debate publicly the
methods and interpretations of published studies.
The major issue raised by intramural research is not, therefore, that
of peer review. Rather, it has to do with the effects this approach
could conceivably have on the quality and content of the entire body
of knowledge of health services research. To the extent that intra-
mural research is used as a substitute for extramural research, the
types of problems addressed and approaches used are determined increas-
ingly by federal personnel, relegating the research community's roles
to those of occasional advisors and critics of published results.
The long term consequences of this strategy would have serious dele-
terious effects on the types of research done and on its quality. As
noted earlier, the committee believes that there must be opportunities
for replication of studies in health services research to guard against
*However, in instances where large-scale studies are undertaken intra-
murally, the committee encourages the use of advisory groups to assist
in their designs, to oversee their implementation, and to review their
findings and interpretations.
OCR for page 70
70
basing health care policy decisions on only a few studies of complex
questions. Given the pressures within the federal government toward
standardization of definitions and methods, a totally intramural
strategy for health services research could greatly impair the process
by which previous research results are subjected to revision by the
appearance of contradictory findings based on different conceptual
and methodological approaches. In the same vein, an exclusively
intramural strategy would, in effect, create a government monopoly
over data that would contribute to the lack of opportunities to
challenge results. Ultimately, the consequences of this strategy
would be seen in the migration of qualified and interested researchers
to other fields of inquiry and the destruction of the infrastructure
of the field of health services research. Without this community
of researchers and their work to draw upon, the quality of federally
supported intramural research would surely decline.
The question of where to draw the boundaries between intramural and
extramural efforts in health services research might be addressed in
terms of the contents of research questions and the government's needs
for information.~6] The intersection of these features in Figure 6
creates four types of situations and identifies funding strategies
most suited to each.
Situations A and B include studies of questions for which relatively
well-established and codified conceptual and methodological solutions
exist, for instance, the enumeration of physician visits using house-
hold surveys. Situations C and D, by contrast, involve problems
for which there are no standard solutions, as for instance questions
about the economic value of life. The columns distinguish situations
in which needs for information are either highly targeted or routine
(A & C) from those in which needs are less well-defined or predictable
(B & D).
The cells of Figure 6 indicate the types of funding strategies that
the committee believes best fit these situations. Intramural research
is best suited to deal with the problems for which standard solutions
exist and for which there are high specific or routine needs for infor-
mation. Most studies performed by the National Center for Health
Statistics are in this category. When needs for such information are
less specific or routine, either the contract mechanism or the intra-
mural strategy is suitable. Here, the contract might be used as
an adjunct to or extension of intramural activities. Problems for
which no routine solution is available call for competitive proposals,
either for contracts in circumstances where needs for information are
specific or routine or for grants in other situations.
Finding that federal agencies are increasingly relying upon intramural
research and research funded by contract, and being concerned over the
long-term consequences of these funding strategies for the types of
OCR for page 71
71
FIGURE 6
RESEARCH FUNDING STRATEGY
Needs for Information
Targeted or Routine
Well-
established
State-of-
the-art
Not well-
established
Not Targeted or Routine
A
B
Intramural Contract
or
I Intramural
...
C D
Contract Grant
.
OCR for page 72
72
research that will be done and for the quality of health services
research, the committee recommends that
the federal government adopt a policy regarding
health services research to assure that a signifi
cant portion of all monies invested in this area
go to support investigator-initiated extramural
research.
-
Intramural research should not be viewed as a substitute for extramural
research, nor contracted research as a substitute for grant-supported
investigator-initiated research. Rather, a strategy of funding should
be developed that identifies the strengths and problems associated with
each and achieves a balance among them.
OCR for page 73
73
REFERENCES
1
2
3
4
5
6
Office of Management and Budget, "Special Analysis L:
Analysis. Budget of the United States Government, 1979
(January 1978),
Healthy Special
U.S. Department of Health, Education, and Welfare, National Institutes
of Health, Office of the Director for Program Planning and Evaluation
Dollars for Health Research and Development: 1968-1975, DREW Pub.
.
no. (NIH) 77-1185 (June 1977):
John Craig, Mega McDonald, and Betty Dooley, "Private Foundations' Health
Expenditures: A Survey Analysis," Health Policy Research Group, George-
town School of Medicine, May, 1978.
Report by the Director, Division of Budget Analysis, DREW, January
1978.
U.S. Congress, Senate, Committee on Appropriations, Departments of
Labor and Health, Education, and Welfare and Related Agencies
-
Appropriations Bill, 1978, Report no. 95-283, 95th Cong., 1st sees.,
1977, p. 81.
Gerald Gordon, Ann E. MacEachron, and G. Lawrence Fisher, "A Contin-
gency Model for the Design of Problem-Solving Research Programs: A
Perspective on Diffusion Research," Milbank Memorial Fund Quarterly/
Health and Society 52 (Spring 1974~: 185-220.
OCR for page 74
Representative terms from entire chapter:
financing administration
