The Productivity of Scientific and Technological Research
SUMMARY
Innovation—the process of converting inventions, ideas, or concepts into commercial products or processes—has always been a convoluted process, but today it is becoming even more difficult to understand and predict. Seemingly minor developments can have major consequences, producing a nonlinearity that defies forecasting. Developments in one field can heavily influence other fields, creating multidisciplinary networks of cause and effect. New ideas can come from anywhere in the production process, not just from the basic research that traditionally has been seen as the driver of innovation. In such a fluid, interconnected system, policy-makers need to create the optimal environment for innovation and then stand back and let the system do its job.
The effectiveness of scientific and technologic innovation depends on many factors in research organizations, including the management and review of research programs, the policies and procedures that apply to those programs, and the broader environment and culture of research. Federal options to improve this effectiveness include the following:
The Research Environment and Culture
-
Increase the size and duration of project awards so that researchers spend more time doing research and less time ensuring that their research is supported.
-
Increase the diversity of the individuals and organizations doing research.
-
Fund risky projects that could dramatically advance an area of research or open new research frontiers.
-
Develop a new digital cyberinfrastructure to make the best use of rapidly expanding databases and multidisciplinary collaborations.
-
Expand funding for merit-reviewed, cross-disciplinary, collaborative research centers.
Program Management and Review
-
Ensure that federal agencies include research programs in their strategic plans and that they evaluate the success of those programs in performance reports.
-
Evaluate research in terms of quality, relevance, and leadership. For basic research, include assessments of the historical value of basic research in contributing to national goals.
-
Evaluate how well research programs develop human resources and the quality, relevance, and leadership of the programs.
-
Establish a formal process to identify and coordinate areas of research that are supported by multiple agencies, and designate a lead agency for each such field.
Administrative Policies and Procedures
-
Develop a new framework for the development of policies, rules, regulations, and laws affecting the partnership between the federal government and the institutions that perform research.
-
Raise the cap on reimbursement of indirect costs to reflect the costs to universities of conducting research.
-
Expand and enhance the Federal Demonstration Partnership to enroll more institutions and heighten the visibility of this important initiative.
THE RESEARCH ENVIRONMENT AND CULTURE
Because innovation does not have a single obvious pathway to success, much depends on the environment and culture that make innovation possible. These factors range widely across social, administrative, and tech-
nological dimensions. The social factors include such considerations as commitment, collaboration, communication, the treatment of multiple viewpoints, workplace diversity, and the willingness to take risks. Administrative factors include salaries, benefits, workplace conditions, the availability of sabbaticals, and travel funding. Technological factors include technical support, training, access to high-speed computing and communications, information services, and so on.
Each of these environmental and cultural dimensions can itself be the subject of innovation. This is most obvious with regard to information technology. To take just one example, a Web site called InnoCentive (www.innocentive.com) now allows companies to post R&D problems online and offer scientists financial rewards for solutions.
The consequences of innovation extend into the social and administrative spheres. For example, increasing the number of women in the biomedical sciences helped focus attention on women’s health issues, with corresponding increases in research in these areas. Similarly, funding researchers at different stages in their careers and at different types of institutions can expand the range of viewpoints brought to bear on a problem.
The federal initiatives that could improve the research environment and culture are unlimited. Among those suggested are the following:1
-
Increase the size and duration of project awards so that researchers spend more time doing research and less time ensuring that their research is supported (see Figures RP-1 and RP-2).
-
Increase the diversity of the individuals and organizations doing research.
-
Fund risky projects that could dramatically advance an area of research or open new research frontiers.
-
Develop a new digital cyberinfrastructure to make the best use of rapidly expanding databases and multidisciplinary collaborations.
-
Expand funding for merit-reviewed, cross-disciplinary, collaborative research centers.
-
Collect the best practices and attributes of federal agencies and research performers and disseminate this information widely.
-
Develop a common electronic grant-application system that combines the best features of current systems and can be used by all researchers and all federal agencies.
1 |
National Science and Technology Council, Business Models Subcommittee. “Comments from the Request for Information.” 2003. Available at: http://rbm.nih.gov/fed_reg_20030906/index.htm. |
PROGRAM MANAGEMENT AND REVIEW
In an era of innovation, the innovation process itself needs to be the subject of research and development. Federal policies that influence scientific and technological research and the commercialization of that research need to be continually re-examined and improved. Valuable sources of insight include international comparisons, the results of small-scale experiments, lessons from other sectors of the economy, and clear, data-based thinking.
One useful way to improve the effectiveness of research programs is by setting goals for those programs and then monitoring the ability of programs to achieve those goals. This was one of the aims of the 1993 Government Performance and Results Act (GPRA), which was designed to encourage greater efficiency, effectiveness, and accountability in federal programs and spending. The act required federal agencies to set strategic goals for at least a 5-year period and then measure their success annually in meeting those goals.
For agencies that support research activities, implementing GPRA has presented many challenges.2 Applied-research programs, whether conducted by federal agencies or private companies, have desired outcomes that are directly related to agency or company missions. Evaluating such programs is therefore relatively straightforward. A series of milestones that should be achieved by particular times can be established, and periodic reporting can indicate progress toward those milestones.
But the usefulness of new basic research is inherently unpredictable. Though history abundantly demonstrates the tremendous value of basic research, the practical outcomes of such research can seldom be identified while the research is in progress. Furthermore, misuse of measurements for basic research could lead to strongly negative results. Measuring this research on the basis of short-term relevance, for example, could be very destructive to quality work.
For both basic and applied research, there are meaningful measures of quality, relevance to agency goals and intended users, and contributions to world leadership in the relevant fields. These measures can be regularly reported, and they represent a sound way to ensure that the country is getting a good return on its research investments. A full description of an agency’s goals and results should contain an evaluation of all research activities and their relevance to an agency’s mission.
Evaluating basic research requires substantial scientific or engineering knowledge. Evaluating applied research requires, in addition, the ability to
recognize its potential applicability to practical problems, which typically requires input from potential users. Expert review should be used to assess both basic-research and applied-research programs. A balance must be achieved between having the most knowledgeable and the most independent individuals serve as reviewers.
Pluralism is a major strength of the US research enterprise. But better communication among agencies would enhance opportunities for collaboration, keep important questions from being overlooked, and reduce inefficient duplication of effort. Identifying a single agency to serve as the focal point for particular fields of research could bring needed cohesion to the federal research effort. In some cases, it may make sense to adopt the model used at the Defense Advanced Research Projects Agency (DARPA), in which the desired end product or technology is defined before research begins, so that research teams can coordinate their efforts to solve the problem.
To improve the effectiveness of federal research and development programs, the federal government could:
-
Ensure that federal agencies include research programs in their strategic plans and that they evaluate the success of those programs in performance reports.3
-
Evaluate research in terms of quality, relevance, and leadership. For basic research, include assessments of the historical value of basic research in contributing to national goals.
-
Evaluate how well research programs develop human resources and the quality, relevance, and leadership of the programs. If federal research activities do not continue to produce a flow of well-educated scientists and engineers, the capability of an agency to fulfill its mission will be compromised and the knowledge learned and technology developed will be lost.
-
Establish a formal process to identify and coordinate areas of research that are supported by multiple agencies. A lead agency should be identified for each such field, and that agency should be responsible for ensuring that coordination occurs among the agencies.
-
Investigate and experiment with innovative ways of managing research, such as establishment of long-term research goals, very flat management structures, multidisciplinary teams, and a focus on technology transfer (these are some of the approaches that have met with considerable success at DARPA).4
ADMINISTRATIVE POLICIES AND PRACTICES
The performers of research sponsored by the federal government operate under an increasing number and variety of administrative requirements. Examples include rules for human subjects, animal welfare, conflicts of interest, costing and administration, agency-specific requirements, and indirect costs. While each rule has its own history and justifications, the combination of often poorly coordinated requirements imposes a significant burden on research performers.
Two publications from the Office of Management and Budget (OMB)— Circular A-21, Cost Principles for Educational Institutions, and Circular A-110, Uniform Administrative Requirements for Grants and Other Agreements with Institutions of Higher Education, Hospitals, and Other Non-Profit Organizations—form the framework for current cost and administrative regulations. Both are in need of revision. In 1999, the National Science and Technology Council (NSTC) released a report titled Renewing the Federal Government-University Research Partnership for the 21st Century, which laid out a set of guiding principles to provide a framework for the development of new policies, rules, regulations, and laws. These principles could be used to define acceptable standards for the conduct of research that could identify areas of deficiency and foster an appropriate balance between compliance with regulations and administrative flexibility.
A particularly contentious issue for college and university researcher performers has been the 26% cap on reimbursement of administrative costs imposed by the federal government in 1991.5 Currently, about a quarter of federal funds spent on research at universities reimburses indirect costs. The two major components of indirect costs are for the construction, maintenance, and operation of facilities used for research and for supporting administrative expenses, such as financial management, institutional review boards, and environment, health, and safety management.
As the administrative demands on universities have increased, these institutions have had to pay for an increasing percentage of indirect costs that are not covered under the 26% cap. As a result, universities have had to shift funds to cover administrative costs from other sources, including tuition, endowments, or state appropriations. Eventually, this cost shifting will be detrimental to the health of these institutions, resulting either in less research, higher tuitions, or reduced services to students.
A more flexible and responsive relationship between federal agencies and universities could help control the administrative costs of research. In 1986, the program now known as the Federal Demonstration Partnership
(FDP) was established to examine, streamline, and reduce the burdens of grant administration. The goals of the FDP are to standardize terms and conditions across federal agencies, simplify the prior-approval process, and streamline award distribution—for example, the FDP is doing a long-term study of institutional burdens related to the OMB circulars. Extending the FDP to colleges with less involvement in federal research awards would help disseminate best practices among federal agencies and institutions of higher education.
Among the actions the federal government could take to reduce the administrative burden on the performers of research are the following:
-
Use the “Principles of the Federal Partnership with Universities in Research” developed by the NSTC to provide a framework for the development of new policies, rules, regulations, and laws affecting the government-university partnership.
-
Raise the cap on reimbursement of indirect costs to reflect the costs to universities of conducting research.
-
Expand and enhance the FDP to enroll more institutions and heighten the visibility of this important initiative.
-
Streamline and align the grant-administration process across agencies to the extent that is consistent with agency needs; all agencies should use uniform terms and conditions for all research and research-related project grants.