Promoting Innovation: 2002 Assessment of the Partnership for Advancing Technology in Housing (2003)

The current federal interest in performance evaluation is driven in large part by the Government Performance and Results Act of 1993 (GPRA). The act requires federal agencies to develop strategic plans, performance measures, annual performance plans, and performance reporting. However, creating performance measures for R&D-related programs has proved very difficult. The nature of the innovation process and the number of factors that influence the outcome of R&D projects make it very difficult to identify performance indicators. The extended time between initiation and measurable results increases the complexity of the problem. The amount of money spent on R&D has been used as a measure, but it assesses only the level of effort and does not evaluate results achieved. Output measures need to be defined for specific research endeavors; evaluation of outcomes often depends on qualitative peer review (GAO, 1997).

The evaluation of PATH is complicated further by its dual role in fostering R&D on new technologies and increasing diffusion of existing technologies. The committee’s assessment of the PATH program critiques its implementation and the effectiveness of its activities to date and makes recommendations for its continuing evaluation and improvement.

GPRA specified that performance evaluation should incorporate the following elements:

1. Performance goals to define the level of performance to be achieved by a program activity;

2. Goals expressed in an objective, quantifiable, and measurable form;

3. Resources required to meet the performance goals;

4. Performance indicators used in measuring the outputs and outcomes of each program activity;

5. A basis for comparing actual program results with performance goals; and

6. The means to verify and validate measured values.

The committee’s 2002 assessment of PATH is based on information provided by HUD and PATH participants (see Appendix C for a list of presentations). It relies on the judgment and expertise of the committee, whose members have expertise in a broad range of housing issues and the application of housing technology (see Appendix A for biographies of committee members). Except for PATH budget data, all of the documentation of PATH activities reviewed by the committee is available on the Internet at either the PATHnet or ToolBase Web sites (HUD, 2002; NAHBRC, 2002a). Though performance metrics of PATH activities and statistical analyses of innovation in the housing industry are needed for a truly rigorous program evaluation, they were not available. Data on the rate of development and diffusion of new technologies are also needed to make the best use of Rogers’ innovation paradigm to better understand innovation in the housing industry and observe the impact of the program.

Because analytical data were not available, this evaluation relies almost entirely on opinion and anecdotal information gathered from discussions with people involved in the housing industry. Given the limits of time and resources available to a volunteer review committee, it did not collect new data by surveys or structured interviews. The committee believes that HUD should collect such data in the future as part of a continuous, comprehensive assessment of PATH activities and their effects, and to respond to all the performance evaluation elements specified in GPRA.

This evaluation assesses selected activities and addresses progress toward achieving PATH goals. It includes qualitative evaluation of the output of the activities to provide direction for improvement of the program. The committee believes that the value of PATH should be judged on its potential for correcting deficiencies and achieving the outcomes defined by its goals in the future. The assessments of intermediate outcomes (measured change in the development and diffusion of technology in housing) and the ultimate outcome (improved performance of homes and home construction) are discussed in Chapter 5 on long-term assessment.

The committee recognizes the need for PATH to address the continuum of innovation and technology diffusion but, as noted in previous reports, believes that PATH should give priority to removing barriers and facilitating the adoption by the housing industry of new technologies. Though addressed as a separate goal, barrier removal is also inherent in all aspects of technology development and adoption and is therefore a part of the mission statement and of all PATH goals and objectives. The committee considers technology transfer and information dissemination to be integral to achieving all aspects of the program’s goals.

The principal PATH mission is to advance technology that improves housing performance (affordability, durability, sustainability, and safety). This approach is distinct from technology advancement programs aimed at increasing the economic competitiveness of an industry. PATH goals and objectives are segments of a continuum that encompasses stimulating innovation, facilitating the diffusion of technology, and administering a true partnership of government, industry, and academic institutions.

This evaluation considered all of the 56 PATH activities undertaken by 11 private contractors and 7 federal agencies from 1999 through 2001, a few of which are addressed here (all PATH activities are discussed in Chapter 2 and described in detail on the Web at PATHnet.org). The committee chose in this report to discuss those activities it believes likely to have the greatest impact on the program’s current goals, evaluating related activities together. This report describes how well the activities have

been performed and how well they support the PATH mission and goals. The total PATH program is evaluated on both achievement toward each goal and the committee’s perception of accomplishments of the program as a whole.

PATH-sponsored field evaluation, demonstration, and pilot projects generate information that will benefit manufacturers, builders, consumers and most other housing stakeholders by providing examples of how technologies perform and how barriers to diffusion of innovation have been eliminated. Information from the 27 projects so far undertaken, which employ more than 35 technologies, can help manufacturers improve their products (NAHBRC, 2002a,b). This has the potential to significantly increase the rate of innovation in housing by providing data that can reduce the perceived risks of new technologies for all participants. A wide variety of technologies have been used at sites that are representative of regional variation across the country. The program emphasizes single-family homes, which is appropriate for this stage in the program.

The program contributes to achieving the PATH goals of disseminating information and eliminating barriers, but improvements are needed for it to achieve its full potential. The problems begin with the distinctions between field evaluations, demonstrations, and pilots. PATH has defined the three variations, but their applications overlap, which confuses both the meaning and the organization of the program. A single program to gather information and document the use of new technologies in the field would allow for greater administrative efficiency. Discussions on PATHnet.org note that field evaluations are generally limited to a few housing units and a few technologies; larger projects are considered demonstration sites and pilots (HUD, 2002). The committee believes that the focus should be on the technologies and that the program should be organized to elicit reliable, reproducible data on product performance with consistent and concise documentation.

The current documentation of demonstration projects is organized by housing project and builder, with no information to link the experience and outcome at one site to the knowledge gained from another. Engineers, architects, and builders coming to PATH will probably be looking for information about a technology, not a builder—the PATH Builder’s Technology Stories are more appropriate vehicles for spotlighting innovative builders. Documentation for some demonstration sites provides no useful information about the technologies other than that they were used on a given project, e.g., Takoma Village Cohousing (Steven Winter Associates, 2001). The reader can search through several reports to compare outcomes, but this is difficult. The technology inventory fact sheets may be intended to serve this purpose, but they do not currently provide this information; nor is there a reference in the demonstration and evaluation reports. Though projects have been documented at inception, there is little explanation of how they proceed through to evaluation and when the reader should expect to see final results.

The reports could also benefit from further review to address inconsistencies like those found in the Washington Square project report that compared the performance of insulating concrete forms (ICFs) to 2 × 4 frame construction, but then stated that the builder’s alternative was 2 × 6 frame construction (NAHBRC, 2002c). This inconsistency made it difficult for the reader to assess the builder’s decision. In some demonstration and evaluation projects, such as Warren Builders’ Homes, Albertville, Alabama (NAHBRC, 2002d), technologies have been combined (HVAC system and enclosure system) making it more difficult to evaluate a single technology.

There is a definite and immediate benefit from project-oriented public relations efforts for the builder, the technologies, and the PATH program, but the lasting value of demonstration and evaluation activities will come from creating a database of product performance that can be used to compare how effective a technology is across different applications over time. This will require standardizing the protocols for collecting and analyzing both metered data and anecdotal information from surveys and interviews. The performance database of the results of demonstration projects can then be of value to participants in all stages of technology development and serve as the basis of reports for specific channels of communication.

Barriers research is needed to understand the impediments to development and diffusion of technology for housing, as emphasized in the industry implementation plan, which noted the importance of identifying and understanding barriers before implementing strategies or taking specific actions (NAHBRC, 1998). It is noted also in the PATH continuum (operating plan) that “To develop successful activities and services for the housing industry, PATH must understand barriers to housing technology research and adoption. Barriers can be found in the building process; in the economic, social, or political aspects of a housing technology; or in general housing characteristics” (HUD, 2002).

PATH has initiated several projects, among them roundtables and focus groups, technical background (“white”) papers, and surveys, to increase understanding of barriers to innovation in housing; however, the committee believes that the barriers research program has not been well planned and no visible progress has been observed to date.

Though the PATHnet continuum lists seven ToolBase roundtables as contributing to barriers analysis, only one, Housing Innovation and the Appraisal Process, directly addressed barriers to innovation. The others were concerned with more general housing issues, such as demographics and labor supply, senior housing markets, and quality control. Roundtable discussions can be valuable as a first step in barriers research, but collecting anecdotal information in open discussion needs to be followed by more in-depth and coordinated research and analysis.

In its 2001 interim assessment the committee applauded the development of a market survey instrument (NRC, 2002); it is disappointing that no further action has been taken to learn more about consumer response to new technologies. PATH launched a new technology barriers analysis project in September 2001 to explore specific industrial, institutional, financial, and cultural barriers to the advancement of housing technology but the results were not available to the committee. The committee concurs that barriers should be thoroughly investigated to learn more about the most effective channels of communication and identify opportunities to reduce impediments to innovation. This effort should be a priority; its scope should be broader and its schedule shorter.

PATH has two initiatives that deal with regulatory barriers: (1) the NES helps technology developers to address potential regulatory barriers to the acceptance of innovative technologies, and (2) the NCSBCS Streamlining project promotes better management practices for regulation of the siting, design, and construction of all types of buildings throughout the United States. The committee believes that support for these activities is worthwhile, but they are unlikely to remove regulatory barriers to innovation. PATH needs to undertake research to better understand regulatory issues and address them with coordinated programs that also improve the regulatory process at the national, state, and local

levels. As noted in the committee’s earlier reports, the HUD code for manufactured housing could be used as a model for improving local codes. Programs to create tools and educational resources for local code officials should be part of PATH’s comprehensive program to remove regulatory barriers (NRC, 2001, 2002).

Finding: Understanding and removing barriers to the adoption of innovative technologies in housing is key to the success of the PATH program. Removing such barriers will increase the rate of innovation by reducing the time needed for diffusion of new technologies, thereby providing additional incentive for private investment in R&D.

Recommendation: PATH should increase the percentage of program resources allocated to the removal of barriers to the adoption of innovative technologies in housing, plan a comprehensive research program to better understand barriers to innovation, and use the knowledge gained from this research as the basis for effective programs to remove barriers.

Finding: It is important for information on the performance, costs, and benefits of new technologies to be disseminated in a useful format to help remove multiple barriers to innovation. To make the program more effective, the process should include feedback on the decisions that potential new adopters make based on the information they receive from PATH. PATH’s demonstration and evaluation projects have not been publicized adequately, nor has PATH developed and documented the data needed to really help homebuilders, regulators, homebuyers, and other housing industry participants understand new technologies and determine whether they should be adopted.

Recommendation: PATH should expand its program of demonstration and evaluation projects and create a database that details the relative advantages or disadvantages, compatibility with existing systems, trialability,¹ and benefits of new technologies. There should be assurance that the data are accurate, reliable, and comparable. The information should be accessible to all members of the housing industry. PATH should coordinate programs to analyze and interpret the data for the industry, regulators, and consumers.

The technology inventory and technology scan are lists of new technologies at various stages of development that have the potential to increase housing performance. The lists are created from information provided by manufacturers and researchers with no PATH-funded performance evaluation.

Potential adopters use the lists as a technology catalogue; they are also a source of ideas for further innovation and a tool for planning PATH activities, such as demonstration and evaluation projects (HUD, 2002; NAHBRC, 2002a). The committee supports the concept of the technology inventory and scan and believes they can play a significant role in the development and diffusion of innovation in housing.

The PATH technology inventory and scan cover technologies still in the laboratory through more mature technologies that still have significant barriers to diffusion. The inventory list is currently limited to technologies that have less than 20 percent of their potential market share. It is not clear to the committee how market performance is determined other than as part of the data submitted by manufacturers. Whether or not the information is reliable, the committee believes that limiting the lists to 20 percent market penetration is inappropriate. Because of this limitation only innovators and early adopters use the list—a mere 16 percent of the potential audience. By the time middle and late adopters consider a technology that is new to them it would no longer be on the list, so information would not be available from PATH. A more inclusive inventory would facilitate comparison of the relative advantages of newer technologies and more mature technologies that have overcome barriers to diffusion.

PATH documents refer to technologies that are or were on the inventory list as “PATH technologies.” This designation is inaccurate and confusing. Since the PATH program has had nothing to do with the development of such technologies nor even conducted an independent evaluation, the assertion that the technology has an affiliation with the program is not warranted.

PATH documents refer to technologies that are no longer on the list as “graduates,” which is also confusing. The term implies that a series of steps were taken to achieve a specified outcome, but this is not documented and probably not justified for most technologies. The committee sees potential benefit in a PATH brand identity like ENERGY STAR, but a clear definition and program plan are needed before such an effort is undertaken.

The key to making the technology inventory and scan effective tools for information dissemination and technology transfer is consistent, authoritative, and reliable documentation. The technology descriptions need to be improved because the current technology fact sheets are more like marketing documents than unbiased factual descriptions (for example, “well suited for” and “quick installation” are used to describe aluminum-plastic composite water piping and “superior performance” and “ease of installation” to describe composite window frames). The impact of marketing-like language is increased when there is no attribution for the information. Although the disclaimer states that the fact sheet does not constitute an endorsement, it does not inform the reader that it is based on unverified claims of the manufacturer.

The fact sheets are also inconsistent and often incomplete in the level of detail provided, which makes evaluation and comparison of alternative technologies more difficult. For example, for recycled plastic-wood composite deck materials, the fact sheet misses key problems of fire safety that have been published by the University of California Forest Products Laboratory. Such problems might again be due to the reliance on manufacturer-supplied information.

Technology scan information is provided in the form of a fact sheet for selected categories of building systems, consisting of a very brief (100 to 150 words) description of the new technology. Though the scan is potentially a valuable tool for technology transfer and stimulation of innovation, more extensive documentation is needed before it can be effective. The current plan for R&D presentations for prospective manufacturers seems very inefficient and the committee believes the current emphasis should be on more detailed documentation and broad dissemination of information.

Nearly half of PATH’s expenditure on information dissemination went to ToolBase, making it the largest non-R&D activity in the program’s budget. The amount of funding and the success of ToolBase highlight both the strengths and weaknesses of PATH.

ToolBase is intended to be the housing industry’s primary resource for technical information on building products, materials, new technologies, business management, and housing systems. The NAHBRC runs ToolBase with funding from PATH and some industry sponsors. The Web site has a daily average of 900 visitors who spend about 9 minutes there. The uniqueness and purpose of these visits are not known. Related ToolBase services are a bimonthly technical newsletter, industry roundtables, e-mail and telephone hotlines, and a biweekly electronic news service.

PATHnet is intended to be an information source for a broader range of housing innovators and adopters. There are no Internet user statistics for PATHnet, but it is frequently referenced on Internet search engines. PATHnet has been allocated only a fraction of the funding provided for ToolBase but it provides effective access to information from PATH and related programs (HUD, 2002; NAHBRC, 2002a).

The information presented on the Internet highlights PATH’s general lack of adequate evaluation mechanisms; consequently, it may lead consumers to products that will fail prematurely. For example, ToolBase has good information about the failures and class-action lawsuits that have plagued hardboard siding but not about similar failures in fiber-cement roofing materials (e.g., Cemwood). The committee noted in its 2000 assessment and 2001 interim assessment that ToolBase should be continually evaluated. Because ToolBase and PATHnet are key programs for disseminating information, it is essential that the information be relevant, unbiased, accurate, clear, and concise. The committee believes that both Internet portals would benefit from continuing independent review.

PATH publications, trade journals and media articles, and conferences and workshops can be effective conduits for information dissemination and technology transfer. A recent survey of West Coast builders indicated that for them, product suppliers, trade magazines, and other builders were the most frequently used channels of communication for information on new technologies, while the Internet was ranked only tenth (Fell et al., 2002). Given PATH’s limited financial resources, it is critical that its funding be devoted to the most effective forms of communication. Resources should be allocated so as to have the greatest possible impact on achieving the program’s mission. Much as ToolBase is the NAHB resource for technological information, NAHB-sponsored Builder is the housing industry’s resource for leading-edge development in homebuilding. The committee notes that while $1.3 million of the 1999 through 2001 budget were spent on demonstration and evaluation projects, exposure in trade and consumer-oriented publications was limited and ineffective. PATH should devote more of its resources to disseminating information in the full range of publications that reach its varied audiences.

Five trade publications—Builder, Journal of Light Construction, Fine Home Building, Construction Specifier, and Architectural Record—serve as primary sources of information for the housing industry and help set opinions for our nation’s builders, architects, and specifiers. An informal survey of their editors produced a surprising and disappointing response. The editors-in-chief did not understand what PATH is and had never written an article about PATH; two had never even heard of PATH. The editors who were familiar with the program said that the only information they received on PATH were boilerplate press releases that did not provide information they deemed useful for publication. The

editors generally thought that if they were provided with unbiased case studies or technical field data describing how an innovative material or system was used to save time, money, and labor, or improve performance, they would publish this type of information every time it was submitted. The committee notes that successful communications through placement of articles in trade magazines depends on long-term relationships with editors, which take both time and effort to build.

Professional Builder, a magazine distributed free to the home construction industry, and its related Web site HousingZone.com have responded to PATH and NAHBRC press releases and published numerous articles on PATH and PATH-sponsored activities. The articles are generally too brief to provide detailed information but they do provide valuable exposure to the program. The committee is unable to judge the impact this magazine and its Web site have on the housing industry and consumers.

When decisions are made about how to disseminate information, the following should be considered:

• Does the communication format improve the technology transfer, development, and diffusion? Is there a real time advantage to a technology-based platform like the Internet over printed documents?

• Is there a direct correlation between resources spent and the success of the communication? If so, is increased funding for the PATH program the only way to ensure that the goals are in fact achievable?

• Will this communication, even if successful, help remove barriers and facilitate technology development and adoption?

In an ideal world, PATH would have enough funding to be able to use publications, outreach, and electronic resources together to introduce new technology to the residential construction industry. The barriers to innovation would be removed, facilitating the development and adoption of these technologies, and everyone from the do-it-yourself homeowner to the chief operating officers of the top five builders would look to PATH for information on the most current technologies for their next project. This means PATH should allocate resources to disseminating information in ways that will most significantly promote the transfer, development, and adoption of technology.

One very successful model for technology transfer is the USDA Cooperative Extension Program. In successfully delivering research data to the field, this program has effectively improved productivity in the agriculture industry. Its decentralized structure, especially its linkages with state land grant universities, helps it meet the needs of a diverse and widely distributed audience base. It is well qualified to address housing issues; a nationwide extension program, created through an interagency agreement between USDA and EPA, is already educating consumers about indoor air quality in homes in over 2,800 counties throughout the United States and its territories (USDA, 2002).

Finding: PATH-sponsored activities like the technology inventory and technology scan can be effective in disseminating information, transferring technology, and planning PATH programs. The current focus on technologies that have achieved less than 20 percent of their potential market share hampers PATH’s effectiveness. The effectiveness of the program is further diminished by the inadequate quality and consistency of materials documenting new technologies and opportunities for technology transfer.

Recommendation: The technology inventory and technology scan should be broadened into a database

of information on housing technologies at all stages of development. The database should incorporate information gained from demonstration and evaluation projects as well as all performance data available. Steps should be taken to ensure that the data are complete and accurate, and that documents used to convey this information to PATH’s audiences are clear, concise, and unbiased.

Finding: Effective communication for the development and diffusion of technology in housing continues to be one of the major opportunities and one of the major obstacles for PATH. PATH uses the many channels and means of communication available, but with varying degrees of success. The current funding for communication is not consistent with its role in achieving the program’s mission and goals. A better understanding of channels of communication that might prove useful is needed to determine the most effective channels and means of delivery. PATH is, again, responsible for ensuring that the information it provides is unbiased, accurate, and complete.

Recommendation: PATH should place more emphasis on and dedicate more of its budget to understanding how its various audiences obtain and use information and to delivering its information. Use of the Internet should be continued, but the use of other means of mass communication and outreach should be expanded commensurate with their role in the housing industry. A process for independent peer review should be created to ensure the accuracy and clarity of the information disseminated.

PATH has conducted four roadmapping sessions, directed by the PATH Industry Steering Committee on information technologies in home construction, panelized-type construction, systems-oriented design and construction, and energy efficiency in existing homes (NAHBRC, 2000). Reports have been published that document these four brainstorming sessions. The program has produced valuable information and insights into the selected technologies but it is unclear how the topics will be combined to develop a research agenda for PATH.

The information provided on PATHnet categorizes the research activities conducted through the NSF as background research, also referred to as basic research. This classification is confusing because most of the funded projects are clearly applied or developmental research.² Whatever the type of research, however, the program has been successful in engaging faculty and graduate students in 22 universities in research related to housing technologies (HUD, 2002).

A review of the project titles reveals a few potentially useful but narrowly focused technical

projects. Although the 2002 projects responded to the PATH roadmapping exercises, there does not appear to be a coordinated program of research that is likely to make much difference in meeting critical housing needs. Some projects appear to be working on problems whose solutions have already been published (e.g., design details to prevent ice build-up in eaves, time and motion studies of brick-laying).

Lack of a mechanism to document and disseminate the information gained from these research projects prevents the program from significantly promoting the development and diffusion of new technologies. The program needs to set a research agenda of issues to be addressed by background (basic) academic research.

PATHnet lists three activities as applied research. They all engage other federal agencies—FEMA, DOE, and USDA FPL (HUD, 2002). Again, the classification is confusing because most of the federal agencies are not undertaking applied research and the activities are only part of PATH-sponsored applied research. Moreover, PATH funding has been applied to activities already in place, making it difficult, if not impossible, to determine the impact of the PATH program. Nevertheless, the activities have produced new knowledge, reports, and demonstration projects that contribute to the PATH mission and goals.

PATH categorizes funded research undertaken by private organizations and corporations as technology development. These projects, which account for more than a third of PATH program funds, are a mix of product development and performance measurement. More than half of the funds are allocated to the NAHBRC; the rest support programs at the MHRA, NASFA, and NIST.

The NIST program transfers about a quarter of the PATH developmental research funds to six private corporations as one-to-one matching funds for the development of new technologies, among them three types of insulated panels of concrete and wood, an automated thermostat, insulated photovoltaic roof tiles, and computer-controlled manufactured housing (HUD, 2002). The committee has not seen the results of these undertakings but, as noted in the 2000 assessment, believes that in principle funding development of proprietary technologies should be a very low priority for PATH.

Among the NASFA projects are investigation of corrosion of galvanized steel fasteners (in conjunction with the University of Hawaii), development of construction details for hybrid wood/steel construction, and a cooperative project with MHRA to study the feasibility of steel-framed manufactured homes. The committee applauds the collaborative approach to these projects, especially the collaboration between industry and an academic institution. An MHRA project that investigated moisture problems in manufactured housing resulted in a report on their causes and possible solutions that provides practical guidance for design, manufacture, installation, and maintenance of manufactured homes.

The NAHBRC undertakes a variety of activities. It is evaluating the performance of steel frame construction and insulating concrete forms, preparing a guide for systems design in residential construction, developing a prefabricated shear wall for light-frame construction, and analyzing the market for vacuum-insulated panel construction. Because the NAHBRC projects are ongoing, it is difficult for the committee to determine how the PATH partnership has affected them.

PATH has allocated over $6 million to what it calls evaluation measures. The majority of these funds went to NIST for evaluation protocols for sealants and coatings (HUD, 2002), and to NAHBRC to support its existing program to instill quality management principles in the homebuilding industry (NAHBRC, 2002a). The committee believes that neither of these activities has contributed significantly to achieving PATH’s goals. The NIST program addressed an industry-identified need and provided a forum for establishing protocols and standards, which is an appropriate PATH activity, but significant funding went to a narrow segment of old technology. The NAHBRC program, initiated before PATH with private funding, could continue as a private sector activity.

Finding: More than 80 percent of PATH resources have been allocated to R&D yet there is no agenda that identifies and prioritizes R&D activities. The technology roadmaps, while providing direction for specific technologies, are not a substitute for a PATH research agenda. The result has been a broad array of unrelated activities—and minimal progress toward achieving program goals. For PATH, basic and applied research on new building materials and systems with broad applications is more appropriate than research for development of specific technologies, but private investment in developmental research should be encouraged. PATH needs to set national priorities for coordinating federally funded R&D activities, minimizing duplication, and encouraging partnerships between industry, government, and academia. It is particularly important to recognize that industry investment in research is minimal, and to create a mechanism that encourages industry to invest in housing technology research.

Recommendation: PATH should increase efforts to monitor promising R&D and enhance dissemination of information about leading-edge housing technology. PATH should set a comprehensive research agenda that is coordinated with current research in government, academic institutions, and industry. PATH-sponsored research on housing technologies should emphasize basic and applied research with broad application and the potential to increase the rate of innovation. PATH should foster development of specific new technologies primarily by promoting private investment.

PATH was conceived as a program that would achieve its goals by leveraging resources through partnerships with other government agencies, industry, and academic institutions. Intragovernmental activities were curtailed when the administration changed in 2001; PATH gives limited support for FEMA publications, DOE building technology programs, and the FPL Advanced Housing Research Center. Funding for several projects at the NIST Building and Fire Research Laboratory (BFRL) makes this the most active PATH government partner.

In its 2000 assessment the committee noted that the day-to-day involvement of other federal agencies with larger, more mature programs (e.g., DOE’s Building America and EPA’s ENERGY STAR) tended to overpower PATH and obscure the value added to their work by the PATH program. In its 2001 assessment the committee noted that the reduced involvement of other agencies alleviates this problem but recommended continuing interagency coordination of programs that affect the develop-

ment and diffusion of technology in housing. Current coordination efforts consist of informal communications among government officials; they need to be strengthened.

As directed by Congress, PATH’s relationship with industry is coordinated through the NAHBRC. This has resulted in intense involvement with builders but far less with manufacturers, designers, regulators, consumers, and other housing industry participants.

PATH’s relationship with academic institutions is primarily through the NSF research program and a few institutions where it has funded projects. PATH is also building a relationship with the National Consortium of Housing Research Centers as a means of communicating with academic institutions.

The NSF research program, while needing improvement as noted above, exemplifies a potentially effective structure for a PATH partnership. The program was initiated by PATH and is conducted with matching funds from PATH and NSF. It uses an established NSF process and relationships with academic institutions to further PATH goals. Unfortunately, most of the other partnerships do not follow this model. Other relationships rely entirely on PATH funding or they are using PATH funds to supplement previously existing activities that they would probably continue on their own without PATH funds.

Members of the committee contacted business associates in the housing industry and housing consumers in an informal survey to gauge how aware they were of PATH. The surveys are neither scientific nor conclusive, but absent more rigorous data they provide some substantiation for conclusions drawn from the committee’s personal experience. The committee has found that participants in the housing industry are generally unaware of the PATH program and have not used the results of its activities. One relatively innovative builder noted that PATH has not made its way into common use among builders. The exceptions were staff of larger builders who are responsible for identifying and evaluating new technologies. This suggests that PATH is communicating with early innovators but has not connected with the majority of homebuilders.

The lack of awareness may be because the program is relatively young, or because it has failed to open channels of communication with housing technology stakeholders. For example, there is very limited direct PATH involvement by local sustainable housing organizations and homebuilder associations that could publicize the program and gather information on the adoption of innovations through community outreach. Although its limited budget makes it impractical for PATH program managers to have direct contact with local organizations nationwide, they could be reached through national organizations and trade publications.

In an informal survey of 15 college-level housing educators, all members of the Housing Education and Research Association, 12 were aware of PATH. Of those 12, 8 expressed concerns that the program was exclusively for builders and had no consumer component, and the other 4 could not accurately describe the program. These college professors teach courses to undergraduate and graduate students in family and consumer sciences, natural resources, planning, or political science curriculum. Some are State Extension Housing Specialists. Most of those surveyed also devote some of their time to research on housing. Participants in this informal survey were primarily from nontechnical disciplines; the results of a similar survey of faculty from technical disciplines (engineering, technology, construction, building science, material science, etc.) might be different. Many university researchers from technical disciplines are members of the National Consortium of Housing Research Centers, and PATH administrators have attended consortium meetings and provided updates.

PATH’s participation in the 2002 Excellence in Building Conference and Exposition sponsored by the Energy and Environmental Building Association (EEBA) and the FPL Advanced Housing Research Centers is a positive step for increasing recognition of the program. The EEBA represents over 10,000 professionals influential in advancing technology in housing. However, the committee is concerned that PATH has missed an opportunity to define itself as a leader, guiding the advancement of technology in American housing. PATH presented an Alternative Building Systems series of discussions covering: (1) An Introduction to Alternative Building Systems, (2) Structural Insulated Panels, (3) Insulated Concrete Forms, (4) Earth Building Systems, (5) Straw Bale Construction, and (6) Regulatory Barriers to Alternative Building. The program is neither cutting-edge nor is it likely to advance technology in housing. The PATH presence seems weak compared to the program presented by the DOE Building America program, which includes the results of current research and addresses cutting-edge technologies (EEBA, 2002).

PATH has published several brochures that describe the program adequately in varying levels of detail for different audiences. While the brochures have a role to play in increasing awareness of the program, the best method for increasing awareness is effective communication of high-quality, relevant information on housing technology, as noted in the Goal 2 evaluation.

As required by Congress, PATH published a Strategy and Operating Plan in October 2000 (HUD, 2000). The strategy and plan were directed at achieving the housing performance goals established at the inception of the program. The committee noted in its 2000 assessment that those goals were too broad and influenced by too many factors to be used as performance measures for PATH and that they are equally inappropriate as a structure for program planning (NRC, 2001). PATH staff and the committee addressed this problem in 2001 by revising the PATH goals (see Chapter 2).

The committee is concerned that the operating plan has not yet been revised to align with the revised goals. Many of the areas of concern noted in this report are due to inadequate planning and lack of alignment of the PATH activities and goals. This is to be expected of activities initiated from 1999 through 2001, because the activities were planned using the original goals; however, a revised plan is needed to align future activities with the revised goals and create a high-performance program.

The 2000 program plan noted that PATH would build a framework for evaluating performance and that information would be collected and tracked by contractors with oversight by the NRC committee. PATH has neither collected nor tracked contractor performance information. The committee has helped PATH to create an evaluation framework (see Chapter 5), but the information so far received from PATH contractors has been insufficient to conduct a structured assessment.

Finding: Administration of the PATH program has been inconsistent and has not provided sufficiently strong direction. The committee recognizes that administration has been hampered by the initial selection of goals at the inception of the program that were overly ambitious for the size of the program. Administration has also been hampered by the uncertainty of the program’s future. Unfortunately, the administrative impediments have led to a misplaced emphasis on activities (e.g., developmental research versus information dissemination and barrier removal), and a program that lacks baseline measures and an operating plan to achieve its goals. The development and diffusion of accurate and unbiased information about new technologies would increase both recognition of the program and its

ability to influence innovation in the housing industry. The strengths of the program in engaging diverse stakeholders and in the skills and abilities of the PATH staff are resources that can overcome these problems.

Recommendation: PATH should draft a program plan for achieving its current goals. Research on innovation in the housing industry and channels of communication should be priorities. The information gained from this research should be used to guide writing of the program plan and collection of baseline data for future program evaluation. All stakeholders should participate in the planning process in proportion to their roles in advancing technology in housing. PATH should enhance its relationships with the broad spectrum of housing researchers, innovators, adaptors, and consumers by establishing channels of communication for collecting and disseminating information on housing technology.

The committee has described the goals assigned to PATH at its inception as inappropriate for a small technology-focused program (NRC, 2001). Nevertheless, PATH tried to respond to these goals by funding activities that promised improvement in one or more performance targets. After more than a year, PATH published a document that incorporated these activities into a strategy for improving innovation in housing. The result is a program with some activities that are appropriate for the long-term pursuit of current program goals (e.g., demonstration projects, technology inventory, and ToolBase) and many (particularly in developmental research) that are not. In the 3 years since its inception, PATH has failed to create a firestorm of innovation in housing. However, though there is no proof of attributable improvements, there is evidence of change in the projects of participating homebuilders, the number of visitors to PATH Internet portals, and the number of academic institutions engaged in housing research.

Although PATH’s approach to diffusing innovation using demonstration projects and information dissemination is limited in its effectiveness, the committee believes these efforts should be improved, not discarded. PATH’s current level of performance is in part due to a lack of knowledge about innovations in residential construction (Koebel, 1999). Koebel made his observations before results were available, but the subsequent performance of PATH-sponsored activities and program planning, as already discussed, support his argument.

PATH has engaged many appropriate stakeholders in the Industry Steering Committee and Roadmapping sessions but has made little progress toward achieving its goals, primarily because of inadequate program planning. This planning shortfall is due in part to inappropriate goals, the necessity

of revising the goals, and inconsistent commitment to the future of the program, but it is also due to incomplete information on the development and diffusion of technology in the housing industry. The planning process also needs baseline data on housing innovation and continuous assessment of PATH-sponsored activities.

Finding: PATH started out with goals that were influenced by many factors other than technology and that were somewhat contradictory, not measurable, and inappropriate for a small technology-focused program. Nevertheless, the program made an effort to achieve these goals. The result is an unfocused program, an array of uncoordinated activities, and a misplaced emphasis on R&D for new technologies. PATH has made an effort to refocus its goals on the program’s role in promoting the development and diffusion of technology, but this effort is not yet complete.

Recommendation: PATH should be continued as a program aimed at increasing the rate of development and diffusion of innovation in the housing industry. Its activities should focus on (1) identifying, understanding, and removing barriers to, and (2) disseminating information for, the development and diffusion of new technologies, as well as (3) increasing industry investment in technology development.

EEBA (Energy and Environmental Building Association). 2002. Excellence in Building 2002 Conference and Exposition. Available on the Web at http://www.eeba.org/conference/default.htm. Accessed September 18, 2002.

Fell, D., Hansen, E.N., and Punches, J. 2002. Segmenting Single-Family Homebuilders on a Measure of Innovativeness. Forest Products Journal 52 (6): 28-34.

GAO (U.S. General Accounting Office). 1997. Measuring Performance: Challenges in Evaluating Research and Development. Washington, D.C.: General Accounting Office.

HUD (Department of Housing and Urban Development). 2002. PATH: A Public-Private Partnership for Advancing Housing Technology. Available on the Web at http://pathnet.org. Accessed September 18, 2002.

HUD. 2000. Partnership for Advancing Technology in Housing: Strategy and Operating Plan. Washington, D.C.: Department of Housing and Urban Development.

Koebel, C.T. 1999. Sustaining Sustainability: Innovation in Housing and the Built Environment. Journal of Urban Technology 6 (3): 75-94.

NAHBRC (National Association of Home Builders Research Center). 2002a. ToolBase Services. Available on the Web at http://toolbase.org/index-toolbase.asp. Accessed September 18, 2002.

NAHBRC. 2002b. PATH Field Evaluations and Technologies, Summary Reports. Upper Marlboro, Md.: NAHB Research Center.

NAHBRC. 2002c. Field Evaluation of Technologies at Bruce Davis, Inc., LaPlata, Maryland. Upper Marlboro, Md.: NAHB Research Center.

NAHBRC. 2002d. Final Report for Field Evaluation of PATH Technologies, Warren Builders’ Homes, Albertville, Al. Upper Marlboro, Md.: NAHB Research Center.

NAHBRC. 2000. PATH Technology Roadmapping. Upper Marlboro, Md.: NAHB Research Center.

NAHBRC. 1998. Building Better Homes at Lower Costs: The Industry Implementation Plan for the Residential National Construction Goals. Upper Marlboro, Md.: NAHB Research Center.

NRC (National Research Council). 2002. The Partnership for Advancing Technology in Housing (PATH) 2001 Assessment, letter report, February 13, 2002. Washington, D.C.: National Research Council.

NRC. 2001. The Partnership for Advancing Technology in Housing: Year 2000 Progress Assessment of the PATH Program. Washington, D.C.: National Academy Press.

OECD (Organization for Economic Cooperation and Development). 2001. Workshop on Basic Research Policy Relevant Definitions and Measurement, 29-30 October: Summary Report. Available on the Web at http://www.oecd.org/EN/document/0,,EN-document-47-1-no-20-4631-47,00.html. Accessed September 17, 2002.

Rogers, E.M. 1995. Diffusion of Innovations. New York, N.Y.: The Free Press.

Steven Winter Associates, Inc. 2001. Takoma Village Cohousing Final Report. Washington, D.C.: Department of Housing and Urban Development.

USDA (U.S. Department of Agriculture). 2002. The Cooperative State Research, Education, and Extension Service. Available on the Web at http://www.reeusda.gov. Accessed September 18, 2002.