Administrative and Funding Structure
The congressional request for a new strategic highway research program indicated that those conducting the study should determine the administrative structure and fiscal needs of the program. Given the diversity and scale of the proposed program, no single research institution, or even a small number of institutions, could undertake the research. Therefore, the F-SHRP committee began its analysis of administrative structures with the assumption that the proposed program would be a large, contract research program under a single administrative structure, as was its predecessor, SHRP. This approach will allow the program to take advantage of research expertise in dozens of institutions across the country while maintaining management focus on coordination in a central authority. In this chapter some key characteristics of successful applied research programs are reviewed; specific criteria based on those characteristics for F-SHRP’s administrative structure are set forth; a comparison of potential administrative structures according to these criteria is presented; the administrative structure recommended by the committee is described; implementation considerations are addressed; the funding requirements and mechanism for the proposed research are delineated; and interim work that should be performed prior to passage of the next highway authorizing legislation is outlined.
Organizational Characteristics of Successful Applied Research Programs
Research is the acquisition of knowledge and the development of solutions through a rigorous, systematic, rational process.1 Regardless of the research category (for example, basic versus applied, scientific versus fundamental
technology, long-term versus short-term), the criterion of applying a rigorous, systematic, rational process is common to all research and requires that research programs possess certain management and organizational characteristics. In transportation, one is generally concerned with applied research, whose aim is the implementation of research products to address current or future needs. This research aim suggests certain program characteristics. In the following sections some characteristics of successful applied research programs, which can be applied in various ways and to different degrees depending on the objectives of a particular program, are described.
Mechanisms to Mitigate Influences That Could Bias the Research
Objective: To avoid bias at the macro level (in the planning and management of the research program) and at the micro level (in the award of individual research projects) that could affect the validity or quality of the research.
An organization managing a research program should ideally be free of political interference, regional and social biases, and micromanagement to insulate those conducting the research from efforts by vested interests to predetermine the outcomes. In reality, influences and biases are not completely avoidable, so a research organization must be structured to protect the research from biases that could affect the quality of the research and to balance the influences of various groups. In fact, some influence is desirable: that which directs the research toward important questions and useful goals—legitimate stakeholder influence—is a positive thing, whereas that which biases the research in ways that jeopardize accuracy, quality, or responsible use of funds is detrimental. Mechanisms for achieving the necessary protection and balance at the macro level include establishing a program within an organization that has credibility among various stakeholders and open processes for stakeholder involvement and governance. Protection against biases at the micro level can be achieved through open solicitation of proposals to ensure that the highest-quality proposals are offered and through merit review to ensure that the best proposals are chosen. A balanced group of stakeholder experts should select the winning proposals.
Mechanisms to Ensure the Quality of the Research
Objective: To ensure quality at the project and program levels.
In addition to merit review of research proposals, stakeholder and peer review can be employed for ongoing research programs and individual research pro-
jects. Project panels may follow individual research projects throughout their duration. Within academia, peer review is often exercised at the journal publication stage. Some state and federal programs also use expert groups to perform periodic review or oversight of individual research projects.
Program-level peer review can be accomplished in many different ways, from one-time or periodic visiting committees of qualified experts and stakeholders to standing committees, such as those managed by TRB. The committees that review applied research programs should include both qualified technical or scientific experts and stakeholders; the former should review the quality of the research, while the latter should review its relevance.
Competence in the Type of Research and in Research Program Management
Objective: To ensure that appropriate capabilities are available to meet the unique technical and managerial requirements of the research program.
An institution must have the ability to support the particular type of research program to be conducted. Technical scope, scale, and the way a program is carried out (by in-house staff or through contracts or both) all have implications for the administrative structure and support functions required for an effective program. A research organization must be able to attract and retain skilled, highly educated, and motivated staff. Unique technical and managerial skills are required, including knowledge of specific technical topics in the chosen research areas and understanding of research management and the process of innovation in the industry. Competence should be an obvious requirement, but it is highlighted here because of the potential complexity of the applications involved and the range of competence—both theoretical and practical—that may be needed. There are several practical implications of this characteristic: recruitment, screening, and hiring of staff can be time-consuming and require the involvement of technical experts and managers knowledgeable about research; salaries must be adequate to attract the best talent in a field; and working conditions and resources must be suitable for creative, intellectual work to be performed.
Focus and Stability
Objective: To guarantee adequate dedication of resources to the research and to ensure that the research program can meet commitments to stakeholders and partners with regard to funding and milestones.
In any organization whose principal task is not research (such as FHWA, state DOTs, and manufacturers), the research effort requires some degree of independence from the main operational activities of the organization. This independence extends to the research and development organizational unit, staff, and budget. Even though applied research must be well connected to stakeholders, this does not mean blending their organizations. Otherwise, the budget and talented staff of a research unit would be easy prey to all the pressing matters of the operational elements of the organization.
Applied research programs are usually driven to produce a product within a relatively fixed time frame for a potential user. Achieving this goal requires stability and long-term commitment to the organization. Stable funding allows for continuity of staff, management, and operations that are not tied to political administrations or budget cycles. A reliable, known commitment of funds, not requiring interim financial decisions at the macro level, allows sufficient flexibility for program managers to make appropriate adjustments as the research progresses.
Objective: To ensure the relevance of the research and the consequent effectiveness of the implementation of its results.
Applied research programs are intended to produce solutions, whether in the near or long term. This requires that the programs be oriented to the needs of users and stakeholders. Some programs, particularly those aimed at producing results in the near term, have their agendas set by those confronting the problems or issues. Although there are different mechanisms for soliciting topics, any applied research program should have some process for asking those who must deal with the problem to define the kinds of research that would be helpful, to indicate some priorities, and to monitor progress for potentially necessary changes in direction or focus.
In addition to determining the content of the research program, stakeholders may have a substantive role in overseeing individual research projects. For instance, a panel of experts, balanced in terms of constituencies, might perform this task. A high level of stakeholder involvement ensures focus and relevance and provides the ancillary benefit of educating the panel members on the subject being studied as the research is being conducted. This in turn helps overcome the typical lag between completion of the research and implementation of its results. By maintaining program focus
and affirming relevance, stakeholder involvement helps ensure that the program is on the right track.
The involvement of stakeholders in longer-term research differs somewhat from that in shorter-term research. Most stakeholders are driven primarily by short-term needs and incentives, which are not necessarily good predictors of long-term trends and opportunities. One way of handling this divergence is to have a separate group of stakeholder-advisors for longer-term research. These advisors would be chosen for their ability to think beyond current problems to anticipate needs and opportunities in their industry 5, 10, or 20 years in advance.
There is a creative tension between this characteristic of stakeholder involvement and the necessary independence researchers need to avoid being influenced by biases from vested interests and being absorbed into non-research activities. In applied research there needs to be a balance between maintaining independence from undue influences and other duties while remaining tied to the problem context. Most attempts to eliminate this tension by oversimplifying the situation end in one extreme or another: producing research that is either not relevant to stakeholders or not creative and forward looking (and consequently not relevant in the long term). Maintaining the necessary balance is one of the unique challenges for research managers.
Criteria for F-SHRP Administrative Structure
By tailoring the characteristics described above to the needs of F-SHRP, the committee developed the following criteria for the F-SHRP administrative structure:
The F-SHRP organization should possess essential quality control mechanisms. The organization should be free to choose the best proposals for each part of the research program using open solicitation and selection based on merit. Mechanisms for avoiding biases in the award and direction of research and for balancing interests and perspectives should be instituted. Appropriate review procedures should be employed throughout the conduct of the research, and the organization should have mechanisms for determining whether and when a particular avenue of research should be redirected or terminated and related contracts modified.
The F-SHRP organization should be competent to carry out a large contract research program. The organization must possess experience in managing such
a program and have appropriate administrative and contract support functions. It should also have the ability to attract and retain talented staff and to obtain additional resources (for example, by entering into partnerships with other research programs and accepting loaned staff). The program should be centrally administered with distributed conduct of research to ensure that multiple subprograms and activities are coordinated and remain focused on the established goals and objectives, while taking advantage of the best talent in a wide variety of research institutions across the country and retaining appropriate control by technical experts over technical issues.
The F-SHRP organization should have focused core staff and secure funding over the program’s time frame. A core staff of appropriate size should be as constant as possible throughout the program, while additional staff may be loaned from other organizations or programs. The program should have a reasonably predictable budget, and the organization must be able to manage the budget on a multiyear, program basis, not subject to annual programming decisions or competition with other research priorities.
The F-SHRP organization should have the flexibility to institute stakeholder governance mechanisms. The governance of the program, at both the executive, overall program level and the technical, component program level, should be carried out by stakeholders. A small governing body composed of leaders from major stakeholder communities should provide strategic direction and be ultimately responsible for the awarding of contracts. Panels for each of the four research programs, composed of users and high-level technical professionals in the disciplines covered by each, should provide technical direction and program review. These panels might also establish expert groups to advise them in particular technical areas. These governance mechanisms should exist in addition to the other types of customer and stakeholder involvement described earlier.
Comparison of Alternative Administrative Structures
In the Strategic Transportation Research Study (TRB 1984), several administrative structures are discussed as potential mechanisms for managing the first SHRP. Some of these options, including the one ultimately chosen, are presented here, along with an assessment of their strengths and weaknesses relative to the criteria discussed above. The various administrative options were considered in the context of a focused, discrete program; it is intended that the organizational structure will disband once the research has been completed.
SHRP Administrative Structure (National Research Council)
The National Research Council (NRC) was chosen as the institution best suited to carry out SHRP. A separate unit of NRC was established with a core staff, as well as loaned staff from other institutions, such as state DOTs and FHWA. The organization provided central coordination for a large contract program, was completely independent, and had the freedom to conduct open solicitation of proposals and to make decisions about midcourse corrections to the research agenda. It was able to attract good staff who were dedicated entirely to the program. The budget was not constrained by an annual programming procedure. SHRP was governed by an executive committee of major stakeholders, with advisory committees and expert task groups overseeing the development of four major research areas. At the completion of the research, the SHRP unit of NRC was disbanded.
One drawback of this approach was the start-up and close-down costs (both in financial terms and in intellectual capital) associated with setting up an entirely new, independent organization, along with all the support functions necessary to manage a large contract research program. In the last year of the program in particular, it was difficult to retain valuable, experienced staff. Although successfully handled by SHRP, this difficulty does represent a drawback of the SHRP model.
NRC continues to be a strong option. The institution (through TRB) is experienced in managing a large contract research program (NCHRP) and has the required administrative and contracting support functions. It has the ability to attract talented staff and other resources. NRC is experienced in convening diverse stakeholder groups and balancing various perspectives and interests. It offers the advantage of a reputation for bringing together a broad array of transportation stakeholders in an open and unbiased forum while utilizing access to experts in other fields. Stakeholder governance and external peer review are part of its normal operating procedures. In contrast to government agencies, NRC is much less constrained in certain management practices. For example, it can more quickly increase the size of its staff to support the program and similarly readjust staff size when the program draws to a close. It has greater flexibility and speed in negotiating and awarding contracts. It can fully implement merit-based selection processes. And it can establish stakeholder governance mechanisms using processes based on those employed in TRB’s cooperative research programs and similar to the processes used for typical NRC committees. Among existing private-sector organizations, NRC,
through TRB, is a well-known and trusted organization in the transportation community.
The drawback of start-up and close-down costs will exist for any option given the scale of the proposed program; in contrast to establishing a new private-sector organization, however, use of an existing organization such as NRC would involve lower start-up costs and less delay. If NRC were used, this drawback could be addressed by making use of experienced staff, support functions, and other resources both within NRC (perhaps by using an existing unit rather than creating a new one) and in other organizations, such as state DOTs, FHWA, and universities.
Other Potential Administrative Approaches
Because of the scope and scale of the proposed research, all of the following options assume that F-SHRP could not merely be part of an existing research program, but would need to be an independent program. The assessment provided here is based on how well the characteristics of each of the existing organizational structures align with the criteria set forth earlier.
FHWA’s Research, Development, and Technology Program
FHWA has had a great deal of experience in managing a large contract research program and performing national-level coordination. The organization has the appropriate support functions for these tasks, can attract talented staff, and has some ability to engage other resources. The annual appropriations process, the impact of outside influences, reduced independence and central control of research, and constraints regarding stakeholder involvement in governance are some limitations of this option.
Other Federal Research Programs
Examples of salient federal research institutions include the Volpe National Transportation Systems Center, which is part of the U.S. Department of Transportation; the National Institute of Standards and Technology, which is part of the U.S. Department of Commerce; the National Science Foundation (NSF); and other national laboratories (for instance, those associated with the U.S. Department of Energy). These entities are experienced with large research programs, and, being exclusively research and technology organizations, have staffs and budgets that are independent of other responsibilities. On the other hand, with the exception of NSF, their research is performed predominantly in-house and therefore would make less use of talent and resources from other
institutions than would be the case under the other options. These institutions also face annual appropriations processes and do not employ stakeholder governance mechanisms. They have the further disadvantage, in some cases, of not having established relationships with all the traditional transportation customers. In addition, national laboratories often have a vested interest in finding applications for technologies they have already developed, which may not be suited to the problems posed by stakeholders. NSF’s mission and approach are not well suited to managing a large, coordinated program oriented toward implementable solutions in a relatively short time frame. Its mission is oriented more toward developing new scientific and engineering knowledge, tied only loosely to ultimate implementation. It is also accustomed to giving researchers a degree of autonomy over the content and direction of research that would be incompatible with the criterion of stakeholder governance.
National Cooperative Highway Research Program
NCHRP has experience in managing a large contract research program, has staff entirely dedicated to research administration, and is insulated from outside influences. It has well-established procedures for stakeholder involvement in governance, although they are limited to state DOTs. Most of its contract research is awarded through a competitive process. While NCHRP could possibly add other stakeholders to its governing body, F-SHRP would dwarf, and possibly overshadow, the traditional NCHRP. In addition, the program is developed on an annual basis, making the planning and conduct of multiyear programs difficult.
University Transportation Programs
University programs, whether at a single institution or utilizing a consortium of universities, are generally independent of outside influence, although internal and interuniversity politics can affect research agendas. The programs generally are characterized by a high degree of research competence but are typically less adept at management and coordination. Programs vary in their success at engaging stakeholders, since university researchers tend to prefer total independence from external direction. Peer review is a common and well-respected practice in the academic world, but it is generally applied after the research has been completed, when the findings are published. While universities are likely to be important in conducting the research under F-SHRP, the academic model is not generally well suited to the administrative requirements of such a program.
American Association of State Highway and Transportation Officials
AASHTO would be relatively free of outside influences and has established procedures for balancing the varied perspectives of its own members. However, the organization may be limited in its ability to involve other stakeholders besides state DOTs in the governance of the program. Although AASHTO has always been closely involved with transportation research, the current organization is not experienced in managing a large contract research program and does not have the support functions required for this work. It is reasonable to say that administration of a research program is not properly within AASHTO’s mission.
Another possible option is to employ an existing private-sector entity (whether nonprofit or for-profit) or establish a new one specifically for the purpose of administering F-SHRP. NRC, already discussed above under the SHRP administrative structure, would fall under this category. Establishing a totally new organization would involve large start-up costs, potentially significant learning curve delay, and relatively high risk for such a short program as compared with using an existing organization. In considering this option, the committee decided it would be best to work with a known organization possessing a track record with the transportation industry.
Recommended Administrative Structure
The committee recommends that the chosen administrative structure adhere closely to the criteria described above. All four criteria should be met, but most important is for the structure to employ quality control mechanisms, such as competitive award of research contracts and merit review, and stakeholder governance mechanisms at the overall program level, as well as for each of the four component research programs. The choice of administrative structure for F-SHRP should be made during the interim work stage described below after appropriate analysis of the above options (and possibly others). The details of the mechanisms to be used to meet the four criteria should be developed during the interim stage as well. The organizational design should address the fundamental aspects of the F-SHRP philosophy outlined in Chapter 1. That is, it should support a customer orientation, a systems approach to research, the incorporation of nontraditional research, and coordination with existing highway (and other appropriate) research and technology programs. The commit-
tee points out that NRC meets these criteria and successfully administered the first SHRP.
Implementation Considerations During the Research Phase
While the program proposed in this report is focused on research, research is helpful only if its results are implemented. The committee anticipates that the majority of implementation activities will take place after the program has concluded. However, implementation must be considered from the earliest planning stages of the research. As discussed in Chapter 2, one of the lessons learned from the first SHRP was the need to address implementation more aggressively during the research program. In addition, early research results will be available for implementation well before the research program has been completed. In this section some implementation-related considerations that should be addressed at the appropriate times during the planning and conduct of the research are presented. These considerations are reviewed here because of their potential impacts on various aspects of the administration and funding of F-SHRP, such as coordination across the four research programs and with research programs outside of F-SHRP, decisions about the direction of the research programs as research results become available, and the allocation of funds to implementation-related activities within each research program. While the committee agrees that the requested funding for F-SHRP should be predominantly for research activities, the implementation-related activities described below also require adequate funding, which is included in the committee’s overall estimate of required program resources.
Identification of expected research products and their users—There should be some assessment of what the market will bear, in terms of cost and volume, with regard to products that may result from the proposed research. Implementation activities (demonstrations, test cases, training) should be outlined and human and financial resource requirements estimated. Institutional and other barriers should be identified. Users and stakeholders should be consulted about the feasibility of implementation given the required resources and about how the identified barriers can be overcome.
Engagement of potential users—Potential users should be involved even while the research plans are being developed (see the later discussion of interim work requirements), and certainly during the research program, so that implementation barriers and incentives can be addressed as early as
possible and a cadre of users will be ready to implement the research products as soon as they are available. These early adopters may even begin to test products before they are generally available so that any problems can be resolved. Early implementers can also aid others in their implementation efforts.
Determination of where the long-term responsibility for implementation coordination and facilitation will lie—Implementation will be done by the ultimate users of the research results: state DOTs, local transportation agencies, and other public and private organizations within the highway community. Some national-level coordination should take place, however. In the case of the first SHRP, FHWA was responsible for this coordination and received funds in the Intermodal Surface Transportation Efficiency Act of 1991 to facilitate implementation of SHRP products through training, conferences, demonstrations, and other activities. It is beyond the scope of this report to recommend the administrative structure or locus of responsibility for F-SHRP implementation activities. Nonetheless, in the interest of ensuring that the proposed research yields the greatest possible benefits, the committee urges that this decision be made as early in the program as possible so that implementation will be coordinated effectively from the outset. This activity will also require separate funding, which is not addressed in this report. The delegation of responsibility for coordinating and facilitating implementation should take into account the nature of the topics proposed for F-SHRP, the nontraditional research disciplines involved, and the wide variety of stakeholders and potential implementers in each focus area.
Dissemination of research findings—This report has stressed the importance of stakeholder involvement in the research process. However, it is not usually possible for all stakeholders and potential users of the research to be actively involved. An effective research program needs to include appropriate methods for disseminating information about research results and products to all who may wish to implement them. Effective dissemination should facilitate implementation by communicating its benefits and connecting users to implementation tools, guidance, and resources.
Coordination of research efforts—The interdependence of the various highway research programs has been emphasized throughout this report, largely from the perspective of the actual conduct of the research. Coordination of these efforts also has benefits from an implementation perspective, including fostering creativity and mutual learning, avoiding unnecessary duplication, and leveraging implementation activities and resources.
Testing and evaluation of research findings—One of the most important and often neglected stages of the research and development process is the testing and evaluation of research results. Testing is done to prove and perfect the results or a prototype technology. It is usually performed under controlled or laboratory conditions and often consists of repetitive application or use under various hypothetical scenarios, including testing to failure, where appropriate. For certain highway products, such as materials and roadside hardware, testing is used to develop specifications that are required by the states before a new product can be incorporated into their procedures. Testing and evaluation also includes an advanced stage of testing that is particularly important for implementation and takes place in real-world, or close to real-world, circumstances. In the rush to market and implement research results, these critical stages may be shortened or neglected, thereby undermining successful implementation.
Evaluation and feedback—Effective research should always incorporate an element of evaluation and feedback, creating an iterative process of continuous improvement both for specific research efforts and for the operation of the overall research program. Evaluation and feedback can be applied to the quality of the research itself (for example, through the peer review process mentioned earlier), the effectiveness of solutions developed through the research, and the success of dissemination and implementation efforts. Stakeholder involvement will ordinarily lead to at least informal feedback in these latter two areas. However, formal evaluation and feedback mechanisms are advisable to introduce some objective performance measures into the evaluation and eliminate bias from the feedback (hearing all praise or all complaints).
Funding Requirements and Mechanism
On the basis of the precedent set by SHRP, the committee recommends that F-SHRP be funded by a 0.25 percent takedown from apportionments from the Highway Trust Fund. Since the funding in question would otherwise go to states for highway programs, the support of the states for this funding mechanism is critical. The committee believes a 0.25 percent takedown will be acceptable to state DOTs if the proposed research is clearly oriented toward their needs and governed by committees that encompass their perspectives. Of course, the states would have to agree collectively to such a proposal and actively support it. Using the federal-aid highway funding levels of the Transportation Equity Act for the 21st Century and assuming a reautho-
rization period of 6 years, this recommended funding mechanism can be expected to produce approximately $450 million to $500 million.
Given the relative scope and complexity of the required activities, the distribution of estimated funding across the four research programs should correspond roughly to that shown in Table 8-1. During the interim planning work, detailed cost estimates should be developed and the total funding requirement, distribution, and percentage takedown of the proposed program modified as necessary. In the event that the established takedown does not provide the expected funding amount (because of a change in the length of the reauthorization period or the effects of revenue-aligned budgeting, for example), the scale and possibly the scope of the proposed research would need to be modified. The authority to effect such modification should be vested in the governing body established to provide overall direction to the program.
Interim Work Requirements
Should the highway community support F-SHRP as proposed herein, the time between publication of this report and passage of the next highway authorizing
Table 8-1 Estimated Distribution of F-SHRP Funding
Research Program Goal
Distribution of Funding (%)
To develop a consistent, systematic approach to performing highway renewal that is rapid, causes minimum disruption, and produces long-lived facilities
To prevent or reduce the severity of highway crashes through more accurate knowledge of crash factors and of the cost-effectiveness of selected countermeasures in addressing these factors
To provide highway users with reliable travel times by preventing and reducing the impact of nonrecurring incidents
To develop approaches and tools for systematically integrating environmental, economic, and community requirements into the analysis, planning, and design of new highway capacity
legislation presents an opportunity to develop detailed plans for carrying out the program. AASHTO and FHWA should consider funding and overseeing the development of these plans. The interim work should be guided by a governing body of stakeholders, which may be assisted by advisory groups of stakeholders and experts for each research program. The advisory groups should include experts in the nontraditional areas indicated for each research program. The following tasks could be funded and performed during the interim period to optimize the time and funding available for the research itself:
Review in detail and synthesize the status of existing research and development programs in related areas. International as well as domestic efforts should be included in this review. In particular, the results of the international technology scanning tours sponsored by FHWA and AASHTO should be examined.
Define in more detail the needs to be addressed by F-SHRP.
Design research activities to address these needs.
Identify stakeholders and potential partners for the full-scale research and development program and begin to engage them in the development of the program. International partners should be included where appropriate.
Develop a budget and schedule for the full-scale research program.
Recommend a detailed administrative structure, including how each of the criteria for the F-SHRP administrative structure described in this chapter will be addressed.
Develop initial requests for proposals so that the solicitation process can begin as soon as funding is available. Particular effort should be made to ensure that the solicitation process yields researchers with expertise in the nontraditional research areas discussed in previous chapters. Potential solutions to the problems identified in this report should be drawn from the widest possible range of expertise, including that generally found in other industries.
In the event that this detailed planning work cannot be carried out during the interim period, it will need to be the first step taken once the research is funded.
TRB Transportation Research Board
TRB. 1984. Special Report 202: America’s Highways: Accelerating the Search for Innovation. National Research Council, Washington, D.C.
AASHO American Association of State Highway Officials
CERF Civil Engineering Research Foundation
HRB Highway Research Board
TRB Transportation Research Board
AASHO. AASHO Road Test. Undated, unsigned paper describing the NRC administration of the AASHO Road Test, from SHRP archives.
Bernstein, H. M., and A. C. Lemer. 1996. Solving the Innovation Puzzle: Challenges Facing the U.S. Design and Construction Industry. American Society of Civil Engineers Press.
Bikson, T. K., S. A. Law, M. Markovich, and B. T. Harder. 1996. NCHRP Report 382: Facilitating the Implementation of Research Findings: A Summary Report. TRB, National Research Council, Washington, D.C.
CERF. Partnership for the Advancement of Infrastructure and Its Renewal Through Transportation (PAIR-T). White Paper.
Christensen, C. M. 2000. The Innovator’s Dilemma. HarperBusiness, New York.
Committee on Science, Engineering, and Public Policy. 1999. Evaluating Federal Research Programs: Research and the Government Performance and Results Act. National Academy Press, Washington, D.C.
Committee on Science, Engineering, and Public Policy. 2001. Implementing the Government Performance and Results Act for Research: A Status Report. National Academy Press, Washington, D.C.
Deen, T. B., and B. T. Harder. 1999. NCHRP Synthesis of Highway Practice 280: Seven Keys to Building a Robust Research Program. TRB, National Research Council, Washington, D.C.
English, L. P. 1999. Plain English on Data Quality: Seven Deadly Misconceptions. DM Review. June/July/Sept.
Hauser, J. R., and F. Zettlemeyer. 1997. Metrics to Evaluate RD&E. Research Technology Management, July–Aug.
HRB. 1960. Special Report 55: Highway Research in the United States: Needs, Expenditures and Applications 1959. National Research Council, Washington, D.C.
Menke, M. M. 1997. Essentials of R&D Strategic Excellence. Research Technology Management, Sept.–Oct.
TRB. 1986. Strategic Highway Research Program: Research Plans. Final Report, NCHRP Project 20-20, Washington, D.C., May.
TRB. 1989. NCHRP Synthesis of Highway Practice 150: Technology Transfer in Selected Highway Agencies. National Research Council, Washington, D.C.
TRB. 1994. Special Report 244: Highway Research: Current Programs and Future Directions. National Research Council, Washington, D.C.
TRB. 1996. Special Report 249: Building Momentum for Change: Creating a Strategic Forum for Innovation in Highway Infrastructure. National Research Council, Washington, D.C.
TRB. 1998. NCHRP Research Results Digest 225: Putting Research into Practice: A Synopsis of Successful Strategies and Case Histories. National Research Council, Washington, D.C., June.
TRB. 1999. Special Report 256: Managing Technology Transfer: A Strategy for the Federal Highway Administration. Research and Technology Coordinating Committee, National Research Council, Washington, D.C.