Understanding Research, Science and Technology Parks: Global Best Practices: Report of a Symposium (2009)

Below is the uncorrected machine-read text of this chapter, intended to provide our own search engines and external engines with highly rich, chapter-representative searchable text of each book. Because it is UNCORRECTED material, please consider the following text as a useful but insufficient proxy for the authoritative book pages.

Panel V The Evaluation Challenge and Policy Synergies Moderator: James Turner U.S. House Science Committee Mr. Turner began by acknowledging Dr. Mary Good as a mentor when they both worked for Signal Companies, before it became Allied Signal and was planning a relationship with Oak Ridge National Laboratory. Since then, he said, he has spent almost 30 years studying how federal legislation could promote in- novation. He had worked on the Bayh-Dole acts, the Technology Transfer Act, various antitrust laws, and each reauthorization of the Small Business Innova- tion Research program, and other issues that set the framework for topics being discussed at the symposium. With this background, he said, he had been thinking about how the world of start-ups and the kind of people who go into science and technology parks had changed. The federal government, he said, had paid little attention to the chal- lenge of designing policies to encourage, plan, or evaluate research parks. The symposium has illustrated the energy with which other governments have made this issue a national priority, and he urged more attention domestically on how best to stimulate innovation at a time when the economy is a central issue. The need to understand and nurture innovation Mr. Turner said that he saw two urgent reasons to pay more attention to innovation. First, the United States must understand what is happening at the cutting edge of science, technology, and innovation around the world if it is to 113 

114 UNDERSTANDING RESEARCH, SCIENCE AND TECHNOLOGY PARKS revive its economy and maintain a standard of living similar to what we have today. He noted that the economic stimulus package then being designed in the Congress would be “neither the first nor the last,” but it was not addressing the fundamental framework in which innovation can flourish. “When a new [adminis- tration] comes in,” he said, “it will be obvious that this issue is an important one. Knowing what’s happening at the cutting edge, and how we can magnify that by an order of magnitude over a period of time, will tell us whether we are going to turn this economy around and maintain a standard of living similar to what we have now. This is an incredibly important exercise we’re going through.” The second reason to pay more attention to innovation, he said, is that the world has changed dramatically in the past 10 years. The Internet has allowed business, education, and government to function in a more distributed way, and has enabled unprecedented collaborations between institutions and sectors. This country, he said, is in a position to nurture the companies that are on the cutting edge of technological practice, and this can influence how well the U.S. economy can compete in the future. But little attention has been paid on how best to nurture companies, or how to evaluate the results of those policies. He recalled the roots of his experience in this arena during the Carter ad- ministration when he took part in a policy review of innovation. That was the stimulus for the Bayh-Dole acts, he said, and many subsequent policies of tech- nology transfer and antitrust policies—“until the Internet came along. What we have to do now is figure out how the world has changed, how to communicate better in new ways, how to mine the new data we have, and how to share it using a common language and standards. What will that mean for technology parks, universities, government? If we could have everyone on that same wavelength we would have a tremendously different way of doing business.” The Role of SBIR and State Awards Robert McMahan State of North Carolina Dr. McMahan said that he works as senior science advisor to the governor of North Carolina and executive director of the Board of Science and Technology, which he called “the nation’s first state-level office of science and technology policy.” He said that while he has no direct role at the Research Triangle Park, he lives adjacent to it and “represents a constituency for the research park—those who are responsible for its funding and its promotion at the state level.” He said he would “take a somewhat contrarian view” in the discussion. While most of the participants have focused on inputs to science and technology parks, he said that he would focus also on outcomes and outcome measurements. He said he would use the state of North Carolina to illustrate the complex and 

panel v: THE EVALUATION CHALLENGE AND POLICY SYNERGIES 115 sometimes counter-intuitive relationship of outcomes to inputs, particularly in the case of university research parks. Transforming the Region North Carolina, he said, has been recognized as successful in founding a research park that influenced the economic transformation of its region. Over 50 years ago, a small group of planners began to create structures to leverage innovation resources within the state. At the time, in the 1950s, North Carolina was “desperately poor,” ranking 49th in the country in per capita income. Most of its population was involved in a few low-wage industries: tobacco, furniture, and textiles. A few strong and committed leaders from the universities and the private sector came together to take action. The key question they asked, he said, is the same one people ask today: How do we leverage local innovation strengths to diversify and strengthen our economy? The idea of a science and technology park was then a novel one, but it made sense to these leaders. Planning quickly began through informal discussions in living rooms and offices. The original fundraising was done by a single individual named Archie Davis, who worked at Wachovia Bank. In 1958, Davis raised $2 million from private donations from across the state in 60 days—today’s equivalent of $15 million—which made it possible to create a large and diverse innovation center for North Carolina. The Importance of “Patient Structures” What is distinctive about North Carolina’s approach is that it recognized the importance of “patient structures,” said Dr. McMahan. For the first 10 years, the park made little progress. But bridging institutions designed to institutionalize change, such as the Board of Science and Technology, provided continuing sup- port. That Board also created innovations that have endured and served as models, especially the clusters of research, development, and innovation activities that “contribute enormous economic impact.” The Board, with a long-term mandate, has been able to look ahead far enough to new fields gaining momentum, and urged the state to invest in biotechnology when biotechnology was not yet a fa- miliar term. The biotech cluster is now the third largest such cluster in the nation. RTP, he said, used a planned approach that “differs from other parks that have grown in more organic fashion,” he said, “and underscores the point that states are emerging as key drivers of collaborative R&D.” When looking at the inputs and outputs of the park, however, he reported one significant discrepancy. The state’s investments have been strongly effective in supporting basic research and development and in the creation of innovation capacity. However, he said, it turned out that this high level of academic research support and performance is not matched by the strength of industrial R&D. So 

116 UNDERSTANDING RESEARCH, SCIENCE AND TECHNOLOGY PARKS in terms of broad measures of commercialization and the total number of R&D dollars being spent, the state is somewhere around the U.S. average, or even in the “second tier of states.” The Challenge of Translating Research into “Gazelle” Firms Dr. McMahan said that a closer study of this imbalance is even more reveal- ing. In the United States as a whole, industry dominates R&D spending; for every $15 spent on industrial research and development, about $5 are spent on university R&D. The industrial spending is essential, he said, because it is one of the most reliable metrics for predicting the dynamism of the technology economy. While academic R&D creates innovation capacity, it does not by itself raise eco- nomic output or create the fast-growing “gazelle” firms that spur new growth. This has held true for RTP. While RTP ranks among the top parks in the country in innovation capacity, it ranks 48th in its creation of “gazelle firms”—small busi- nesses with year-over-year growth above 20 percent for four consecutive years. “We see this same trend in emerging domains like nanotechnology,” he said. “We are in the top ten in terms of nanotechnology research, but not in the number of companies being created.” It has been generally assumed, he said, that simply increasing levels of R&D proximate to universities raises the level of economic activity. “But often when you delve into the numbers this is not so. We need to understand structurally what the economic outcome of R&D is in terms of the major drivers of job growth and wealth creation in our economy.” In the United States, a majority of jobs are created by small businesses; of all companies, he said, about 15 percent are responsible for the bulk of job creation. “So the importance of targeting economic growth in that segment is clear.” RTP is now studying how better to convert its innovation capacity into jobs and other economic outcomes. One such effort is called UNC Tomorrow, a comprehensive plan to engage all 16 campuses of the state university system more closely around economic engagement. A Commitment to SBIR Dr. McMahan then came to the topic of SBIR, the Small Business Innova- tion Research program, which targets specifically the high-growth, high-potential companies mostly likely to achieve commercialization.20 Statewide SBIR activity, he said, is an excellent proxy for the dynamism of the “gazelle” population. For this reason, three years ago North Carolina instituted one of the country’s most substantial SBIR support programs as part of its support of ­entrepreneurship. 20  The National Academies has recently concluded the first comprehensive assessment of the program since its founding in 1982. See National Research Council, An Assessment of the SBIR Program, op. cit. 

panel v: THE EVALUATION CHALLENGE AND POLICY SYNERGIES 117 The state decided to award up to $100,000 in matching funds to each SBIR com- pany. Already this approach has succeeded in targeting high-potential small firms, he said, “and should be viewed as a powerful complement of research parks and other structures.” He summarized by reiterating that the simple creation of parks and other organizations is not enough to sustain economic growth and create high-paying jobs. “We have to place as much weight on the environment in which they are cre- ated and the policy framework in which they exist. Otherwise, university-based regional economic activity is insufficient.” This conclusion, he said, “drives us to a policy view of the university-based park as part of a continuum of mixed infrastructure and other measures. We have to ask ourselves constantly, how do we enhance their productivity and success? Entrepreneurship becomes critical to the success of the parks and the larger economy.” The Evaluation Challenge Albert N. Link University of North Carolina at Greensboro Dr. Link, a professor of economics who has long studied research parks in the context of economic growth, said he would offer his perspective on the chal- lenge of evaluating park performance. He said that in 1985 he was asked to begin writing the history of Research Triangle Park, which has taken two volumes to compete. The first volume assembles an oral history, and the second is an account of the park’s history and development.21 The Need for Accountability Dr. Link noted two aspects to the challenge of evaluating a park such as RTP. The first is to identify the “environments” affected by park activities. The second is to identify evaluation metrics that are appropriate to each activity. Once the impacts have been identified, the review must design ways to quantify them and their costs. He began with the question of why parks should be evaluated. His response was again twofold: 21  A. N. Link, A Generosity of Spirit: The Early History of the Research Triangle Park, Research Tri- angle Park, NC: University of North Carolina Press for the Research Triangle Park Foundation, 1995; and A. N. Link, From Seed to Harvest: The Growth of the Research Triangle Park, Research Triangle Park, NC: University of North Carolina Press for the Research Triangle Park Foundation, 2002. 

118 UNDERSTANDING RESEARCH, SCIENCE AND TECHNOLOGY PARKS • Funds are needed for the development of park land and infrastructure, and more than 80 percent of parks rely on government or university funds or both. • It is reasonable to expect that both “investors” and the local community will want accurate evaluations of parks. This strong need for accountability, he said, is reasonable, because park d ­ irectors are stewards of public money, and the public sector has a long history of being accountable for its use of public resources. This history dates back to the Budget and Accounting Act of 1921, which was followed by fundamental budget reforms carried out under President Wilson. More recently, Congress passed the Government Performance and Results Act of 1993 to update the terms of account- ability for federal agencies. Rationale for Evaluating Parks Accountability is an especially important issue for universities, which, he said “are not known to be good managers.” In this context, he suggested that exercises in evaluation actually help universities become better managers by helping them identify all the agents of the university that are affected by the park. In addition, he said, the budget constraints of the current period imply that the legislative initiatives that have supported the creation of parks will have to be justified, which will require at a minimum some in-depth case evaluations of selected parks around the country. “Many individuals in the administration to- day,” he said, “who are affected by the decline in science budgets are suggesting that this is going to set science back quite a few years.” One reason for this, he said, is that the need for the National Institutes of Health and the National Sci- ence Foundation is more apparent than the need of some other science-related activities, and these more visible and popular programs are more likely to grow. Another rationale for park evaluation, however, is that such an exercise often reveals ways to increase the efficiency of the science that takes place, both at the university and at independent companies. The Government Performance Results Act (GPRA), passed in 1993 by Con- gress, has had a demonstrable trickle-down effect, he said.22 States began to examine their own procedures and look for ways to quantify accountability. The universities, he said, also paid closer attention to what faculty do with their time, how much contact they really have with their students, and other activities. The trend toward evaluation is also going to trickle down to the science parks, he said, 22  The Government Performance and Results Act of 1993 required federal agencies to develop long-term Strategic Plans defining general goals and objectives for their programs, to develop ­Annual Performance Plans specifying measurable performance goals for all of the program activities in their budgets, and to publish an Annual Performance Report showing actual results compared to each a ­ nnual performance goal. 

panel v: THE EVALUATION CHALLENGE AND POLICY SYNERGIES 119 b ­ ecause a good deal of public money goes into creating and maintaining them. He pointed out that many European countries evaluate their parks even before they are funded, as well as afterward. Many Kinds of Outcomes He suggested many kinds of outcomes that can justify investments in parks. Among the more apparent are spillovers to the global economy that “usually take the form of the creation of codified knowledge.” Spillovers, he said, can also be examined in other ways. Tenants often form research joint ventures with other firms in the park. Also, tenant companies may throw off benefits to the host university by sponsoring laboratories and professorships, hiring students, or associating themselves with co-patenting activity. (He noted a shortcoming in co-patenting activities, in that it generally results in the patent going “out the back door from the university directly into the company.”) Finally, the local com- munity may induce in-migration of other companies into the area, some of which may locate in the park. Another kind of outcome is the value of the park to tenant companies, which benefit from the richness of the flow of knowledge between them and universi- ties. “The greatest challenge is to keep them in the park for extended periods,” he said. Although it is not common for one park to lure tenants away from another, companies may cut back on their level of R&D, or consolidate their research or- ganizations at a central laboratory. The best way to keep tenants is to ensure that the flow of knowledge continues at a high rate. “This,” he said, “is a challenge for the university and the reward structure of the university.” How Universities Benefit from Parks While parks gain knowledge and prestige from their university partners, the universities gain as well. Association with the park brings the university higher publication rates, more successful patenting activities, greater ability to hire eminent scientists, and larger extramural grants. Interviews with provosts of universities associated with parks say that one result is to shift curricula toward more applied research and to improve the ability to hire graduate students in science and engineering. The university as a whole becomes more interested in innovation. Faculty who mentor students are more attuned to the needs of industry. The change in the structure of the curriculum is reflected by improved student placements. The “Cachet” of a Park Another outcome that may be less obvious, he said, is the cachet of work- ing in a successful park, which can benefit the host university, tenant firms, and 

120 UNDERSTANDING RESEARCH, SCIENCE AND TECHNOLOGY PARKS the local community. He recalled the example of the Semiconductor Research C ­ orporation (SRC), a consortium of U.S. and now international semiconductor firms formed primarily to work out intellectual property agreements with many universities. It was initially located in RTP, but as it began operation, some members were more satisfied than others. A few companies talked about leaving, saying that the park was too expensive. In response, the SRC cut costs by moving out of the park. At the same time, it wanted to maintain the “halo effect” of a park address, so it kept its mailbox at RTP. There is a need, he said, for more studies that compare the performances of firms located in parks with those of comparable firms not in parks. He said that the best study, done in the UK, found that the performance of firms is greatly enhanced by location in a park and juxtaposition to a university. Rewarding Those Who Create Knowledge Dr. Link closed by offering several reflections on park evaluations. For one thing, he said, identifying the stakeholders and the many linkages between them is “easier said than done.” University administrators have a tendency to recognize their own “outward” influence on job creation and other impacts on the local community. But they are less likely to ensure that faculty who create this flow of knowledge between university and firm are rewarded and appreciated. He concluded by predicting that more parks are coming, but that they will need better ways to demonstrate their value. “There are about 100-plus parks that have a formal relationship with a university,” he concluded, “and about another 25 to 30 in planning stage. I think these numbers will grow; not automatically—not ‘if we build it they will come.’ That model applied in the early 1990s but not now. These new parks are going to depend on placement near specific universities and strength in specific technologies.” Discussant William Kittredge U.S. Department of Commerce Dr. Kittredge, who heads the Economic Development Administration’s O ­ ffice of National Programs and Performance Evaluation (NPPE) within the Department of Commerce, said that his office performs several functions relevant to the evaluation of S&T parks. NPPE reviews all proposed investments—such as the fiber-optic system recently installed at Sandia—as a pre-audit function and, in doing so, seeks to advance the science of measurement and outcome prediction. NPPE also performs evaluations retrospectively to assess the outcomes of federal 

panel v: THE EVALUATION CHALLENGE AND POLICY SYNERGIES 121 investments. Dr. Kittredge serves in a similar capacity as U.S. representative to the Organisation for Economic Cooperation and Development (OECD). He said that developing metrics to measure the performance of S&T parks and of economic development in general is a “work in progress.” What we do know, he said, is that people want to evaluate outcomes. They want to know “whether they, their children, and their community are going to prosper and be competitive in the global economy.” But no measurement system will be able to determine policy in this area, because so much of the decision making is “so- cially mediated.” What is successful for the French, he said, may not work for the United States. He also stated his opinion that while outcome measurement informs public policy decisions, it is not dispositive. The Limits of Quantitative Measures With respect to the yearning for quantitative measurements, he quoted a for- mer professor who said, “If you can’t count it, it doesn’t count.” With due respect to the professor, he said, it is apparent that qualitative measures also have an im- portant role in park evaluation because qualitative measures add dimensionality and nuance to the more generalized quantitative measurements. The result is a more complete understanding of the characteristics of successful parks enhancing our ability to make fruitful science and technology park investments. Measurement limitations are also imposed by definitional limitations, he said. More accurate evaluation awaits broader agreement on a definition of in- novation and other key terms. “How do we decide what to accept as metrics?” he asked. For business incubators, it seems natural to measure the private investment and job creation. However, he gave one example of how unreliable this can be. He was asked to evaluate business incubators the department had supported. He found that one particular incubator, six years after it began operation, had listed no jobs and no private investment. What had happened, he said, was that by the time they did their assessment, both the companies had already gone public, so “nobody was there. That was actually a good thing. We need to be careful about what we measure, how we measure, and when we measure” to effect accurate evaluation of science and technology park investments. “Multi-dimensional evaluation that includes private investment, graduation rate, and other key variables is the key to meaningful evaluation and credible communication of our results,” he said. He added that one has to be careful about expectations. In a research park, one might expert to measure success in terms of either serial entrepreneurship or a long-term park presence, whereas a firm in an incubator might be more inter- ested in “graduating” from the park. “Whatever we decide,” he said, “we need a common language for the business community, investors, university, researchers, engineers, and other stakeholders.” 

122 UNDERSTANDING RESEARCH, SCIENCE AND TECHNOLOGY PARKS Some Qualities of Good Metrics Quantitative measures are important, he said, because “I want something I can see.” But they are only one kind of tool. He said that it would be impossible to predict the success of an innovative firm such as Federal Express by using traditional quantitative measures. “Measures need to have certain characteristics,” he said. “They have to be credible and make claims that can be sustained. They have to be transparent, material, contextually important. We have to be able look at our failures, assess them, and accept that it was experimentation that didn’t work. I personally object to the ‘but for’ justification: If we hadn’t done this, this wouldn’t be here. Well, you don’t know that for sure and making the claim undermines our credibility.” “Good” metrics evolve as our expectations and aspirations grow and change. “I understand that some people in the Research Triangle are now focusing on value-added activities that grow from the world-class area we all recognize as a model of success.” The new metrics under consideration seek to retain more value-added activity in the Research Triangle area rather than exporting innova- tions to be implemented elsewhere. In summary, he suggested that a good outcome measure is one that demon- strates the value proposition for stakeholders, and that we are far from having such a measure. “We owe the stakeholders a credible review of what their invest- ment is doing. But I don’t think we’re going to find a single metric for that. Our goal is to increase our knowledge as we go along, knowing that it’s difficult to get cause and effect together. We can get improvements, so we can become more credible; we can understand what good investments are, make more of them, move forward, and expand them. Forums like this are important steps.”