What Can We Learn from Technology Assessment?
EXPECTATIONS OF TECHNOLOGY ASSESSMENT?1
Recently, technology assessment has been receiving more attention than ever before due to the growing perception of technology as something critically important to both national security and industrial competitiveness.
Although the tension between East and West has been reduced dramatically, the Gulf War reminded us of the importance of maintaining military technical superiority as a deterrent to potential military buildup by Third World countries.
Within the nonmilitary economic sphere, competitiveness is a matter of national concern, since employment, the balance of payments in international trade, and national income are influenced by the health of a country's industries. Technology is seen as playing a key role in the development and improvement of products and of manufacturing processes. Therefore, it is natural for citizens to want to know whether their country possesses sufficient technological capability to make its industries competitive in the world.
In light of national security, the health of the nonmilitary industrial sector is also a matter of concern. Most of the important high technology from the nonmilitary sector is applicable in the defense sector, and the
dependence of the latter on the former for such things as the supply of components and manufacturing technologies has increased.
It is natural that governments are concerned about whether their nation's vital industries can compete with the industries of other nations. Further, should such problems exist, governments may try to resolve them by taking measures such as reallocating R&D resources, improving the infrastructure, modifying regulatory settings or tax systems, and creating trade incentives. Technology assessment is, therefore, an important policy tool.
ILLUSIONS ASSOCIATED WITH TECHNOLOGY ASSESSMENT
The essential question at hand is to what extent technology assessment can respond to the concerns presented above. Unfortunately, there are several problems associated with technology assessment that often cause it to be misinterpreted.
First, technology is not necessarily the dominant factor deciding an industry's competitiveness. Too often, people tend to focus on the performance of a product when the technological level is assessed. However, performance is only one of the elements that influence the competitiveness of the product. Price, design, and services at the time of and after the sale of the product are also important factors in competitiveness, although they are not necessarily directly associated with technological sophistication. For components, compatibility is another important factor that determines competitiveness. In the world of personal computers and video recorders, the availability of a variety of commercial software has also been important for success (e.g., in the predominance of VHS over beta). Easy-to-understand operation manuals or textbooks (and available training courses) increase the competitiveness of the products such as word processors and machine tools. Moreover, fluctuation in the exchange rate can easily offset efforts to cut costs or to increase productivity by the accumulation of incremental technological improvements.
Naturally, it is difficult to assess the future technological competence of a product by evaluating the current technological potential or research and development.
Second, how should we define a nation's technological capacity? Increasingly, independent enterprises have globalized their production and research activities, making their substantive nationality obscure. Therefore, the competitiveness of a nation is not automatically equal to that of its domestically owned enterprises. In addition, it has become impossible for an enterprise to depend solely upon original technology. Firms develop products or processes by taking advantage of self-developed technology as well as licensed and purchased technology from other domestic and foreign sources.
Another consideration is that sometimes firms pursue strategic partnerships with their international competitors. For example, many such partnerships have been formed in the automobile and semiconductor industries. In the automobile industry, there are numerous items and stages in the process of designing and manufacturing a car. Therefore, there are many opportunities for a company to cooperate with its competitors—in certain fields and stages of research and development, for example, or perhaps to have another company supply a certain component. Yet, when the end product is sent to compete in the marketplace, it will face products from those very same companies. In the semiconductor industry, technological progress is extremely rapid and requires a large amount of investment. The diversification of related technologies is also rapid. There are tens of thousands of variations of semiconductors, each incorporating various technologies. Also, there often exist several alternative technologies that can be used to produce certain types of products. For example, there are two fundamentally different approaches to create a circuit on a wafer, the stuck method and the trench method. Firms are not always certain which alternative will survive into the next generation of manufacturing, or whether completely new methods will supplant the existing ones. It is impossible for a firm to pursue independently all the variations of such technologies. Therefore, companies try to hedge the risk inherent in the development of new products through strategic partnerships with their competitors. When firms become deeply interdependent internationally in terms of technology, it is difficult to define "national technological capabilities."
TECHNICAL PROBLEMS ASSOCIATED WITH TECHNOLOGY ASSESSMENT
Usually technology assessment attempts to compare the performance of a product made by different countries (or firms), and/or the level of manufacturing or processing technologies of a product. When future technical capability is investigated, the research and development potential of a product as well as the quality and availability of an infrastructure (or any environmental conditions) for its development are also subject to examination. However, there are technical limitations in this assessment that we must keep in mind.
Assessment of Product Performance
When the highest functional performance of a product is in question, as is likely in the case of the evaluation of weapon systems, the performance of the high-end products is usually compared. In this situation, making a comparison is usually not very difficult since the results are likely to be measured numerically. However, should an evaluation that takes economic efficiencies into consideration be conducted, the results might not be so
straightforward. It is likely, for example, that a marginal improvement in the performance would require a substantially larger amount of resources. Therefore, taking into account a limitation of resources—both financial and human—the technological superiority of a certain product may simply reflect the priority of resource allocation. Even in the assessment of the performance of weapons, cost must be a substantial consideration. Needless to say, however, cost consciousness is probably more important for nonmilitary products.
There is also a technical question in evaluating those products that have a wide variation of types. Should products of the high end (the highest performance) be compared, or should we compare the medium (or the largest-volume) products? How should products that are differentiated when firms (or countries) demarcate the market be treated? Articulation of the objectives of the assessment is critical when there exists great product variation. Automobiles are one such area. The origin of variation in automobile production comes from the differences in the concept of designing a car. Fundamental design (or the philosophy of the design) depends on the targeted users. There is a variety of elements that cause product differentiation. Further, there are cases whereby different technological options exist, and since each has advantages and disadvantages, it cannot be said that one is superior to another. For example, in controlling the supply of fuel and combustion in the cylinder of an engine, both electronic and mechanical means are applicable. Depending on the circumstances, it does not automatically follow that an electronic device is technologically superior to a mechanical device.
Assessment of Production Processes
Many factors influence the performance of a product, and there are no universal criteria to weigh all factors together.
First of all, an assessment will differ depending upon whether the intent is to produce high-performance products regardless of the cost or to manufacture products in the most cost-efficient manner.
Secondly, there are many nontechnological factors that significantly influence both the performance and the productivity of the manufacturing process. For example, the motivation of workers is a very important factor even in a highly automated semiconductor factory equipped with the newest instruments and facilities. The yield in such a factory can be greatly influenced by dust (in particular, dust contaminated by sodium ions) created when people enter the clean room. Many firms have created teams to tackle the problem of reducing dust. Not all of the members of such teams are engineers or experts in certain specialties. Teams used average workers to test how dust might be generated by motion, such as walking, speaking, clapping hands, and breathing. Some teams tested the measure of sweat
that might leak from workers' clothes. Through practical research, the teams discovered important know-how. Workers are now required, for example, to change gloves twice a day to suppress the emission of small fiber pieces, and must take a bath and change their underwear before entering the clean room to eliminate contamination by sodium ions. With all of these efforts, there are bound to be variations in the productivity performance of the workers in a semiconductor plant. In other words, there are variations in worker performance that cannot be ensured only through a manual or work policy.
Efforts to improve productivity like those mentioned above are nothing but traditional approaches to troubleshooting. Such troubleshooting is necessary even when the dead copy of a factory (or a production line) in operation is replicated. There is no magic that makes a difference, rather an effort of engineering-focused minds. Within technology assessment, there is a difficulty in measuring the importance of such aspects; indicators (i.e., quantitative measures that could show technological advantage) providing means to evaluate the entire technological value are elusive.
The third point is that newness or technological sophistication of instruments and facilities used in production does not automatically guarantee an advantage in performance; nor does it ensure high productivity in the manufacturing process. In many cases, the human factor plays a significant role. For example, one very competitive machine tool manufacturer uses manually controlled mother machines that have been in operation for decades.
Finally, how should we assess the technological capability of a production process when numerous suppliers, both domestic and foreign, supply very important materials and components? One may argue that key technology materials and components should be assessed within the evaluation of the whole production processes. However the question of what the key technologies truly are arises. Regardless of the technological sophistication of certain materials or components, they may not be attractive to manufacturers if the supply is not stable. On the other hand, when the supply of such components is stable, manufacturers will not be concerned even if those sophisticated materials and components are provided only by a limited number of suppliers. Some materials and components may constitute a part that cannot be replaced by others. Needless to say, it is difficult to assess the impact of this phenomenon when the production process for a product is scattered all over the world. The criteria and the measure of the technological assessment may vary significantly, depending on the emphasis of the conductor of the assessment.
Research and Development Potential
Research and development potential is frequently discussed as a measure to foretell the future technological capability of an industry or country.
First, two fundamentally different approaches exist: namely, evaluation of originality and creativity, and evaluation of a product's application and improvement potential. Which approach is taken depends upon the analyst's judgment and how he or she weighs these factors.
Second, the planning and management of research and development projects are influenced by the strategy of the R&D designers, and whether they choose to emphasize short-term results or to plan for expansion into future possibilities. Firms that seek short-term tangible results may show higher levels of productivity in research and development. However, their long-term success may not necessarily be at the same level.
Third, some companies may place priority on product development, and others on process development. In addition, in certain cases, new product development and new process development must be synchronized; both must be given equal emphasis. This is evident, for example, in the production of miniaturized appliances, such as personal cassette recorders.
Fourth, there are different stages in research and development: basic, applied, and development. Among these, basic research has the character of being "common property" for all nations; therefore, a country's strength in the level of basic research does not necessarily mean that the country will be strong in more advanced R&D stages. However, basic research might be expected to provide a major contribution to the other stages through lending a supply of well-trained and educated experts.
Fifth, similar to the situation in evaluating the production process, many enterprises from different technological areas are involved in the R&D process, particularly in the field of manufactured goods. Domestic firms are not the only participant enterprises.
Finally, many international enterprises perform research and development all over the world. Should the ability of research institutes owned by foreign firms in a country, for example, be counted as part of that country's national R&D capability? It should be counted if the conductor of the assessment is interested in finding out whether or not a nation as a geographical region provides favorable conditions for research and development.
In conclusion, there is no a priori criterion for evaluating research and development potentials. Besides, it should not be forgotten that competition in the market is an important catalyst for research and development. A survey conducted by the Ministry of International Trade and Industry (MITI) three years ago revealed that companies considered competition to be the biggest incentive for R&D (over 70 percent), while limitations in the numbers of researchers and the amount of investment were major disincentives (45 and 35 percent, respectively). In contrast, abundance of researchers and adequate available financial resources worked as an incentive for R&D only modestly (18 and 27 percent, respectively).
There are numerous conditions that influence production as well as R&D activities; for example, the industrial setting, competition among firms, the existence of capable suppliers, regulations, the quality and quantity of the labor force, transportation, electricity and water supply, and the information network. Which factors are selected and what criteria are applied to the evaluation depend on the concerns of the conductors of the assessment.
Problems Associated with Methodology
In addition to the complexity of the technology as discussed above, there are limitations on the process of assessment in terms of methodology. A popular way of collecting data is to categorize technological fields in order to send questionnaires to the experts in selected fields, then to process the responses statistically. For an objective evaluation, it is desirable to compare quantifiable indicators. However, in such questionnaires the field experts are commonly asked to provide their personal evaluations (e.g., by choosing one of three alternatives: superior, even, or inferior) of the technological level or potential of specific products or key technologies. There are two reasons for this method. One reason is that it is not easy to find appropriate objective indicators. In addition, many indicators may not be available to the assessment team because of corporate secrecy. Also there are so many factors which influence technological capability that it is hard to find a way to examine those factors and make a total evaluation. Therefore, the conductor of the technology assessment might conclude that it is better to leave the fundamental evaluation to experts in the field. One problem with such an approach is the fact that the information the experts have could be fairly limited. Particularly, this would be a problem in the event that the products or production process to be assessed involved many different technologies (or industries). In such a situation, the answers of the field experts are significantly influenced by the widely shared views spread by mass media. Whether such views actually reflect reality may not be sufficiently challenged.
Another frequently used method is to make an evaluation based on selected key technologies. However, as discussed previously in this paper, there are no a priori objective criteria to select key technologies. A standard method is to ask the experts to list what they consider to be critical technologies.
A survey conducted for a MITI white paper on industrial technology three years ago followed this approach in assessing 40 technological and product fields. Through interviews with experts, technologies from different industrial fields were selected as key technologies (see Figure 1). It is significant that various technologies might have played key roles in determining the outcome; also the fact that the importance of a specific technology might have changed from time to time should have been considered.
The case of the optical fiber field is one good example. The development of the optical fiber was initiated by glass manufacturers. Then, a cable-making company developed the technology for coating the glass fiber, which successfully enforced its mechanical properties. Next came a new laser technology that allowed wavelengths to be transmitted through this fiber for long-distance communication. This development was followed by the discovery of the vapor-phase axial deposition and the modified chemical vapor deposition methods, which enabled mass production of the fiber. A lesson from this case is that different types of element technology played key roles at different times to advance a whole technology. Another matter of note is that enterprises in different technological fields entered into the development process. It is not always clear which industrial sector will enter into the process of development of a new technology. Besides, as previously mentioned, it is often not certain which technological alternatives will survive into the future.
Indeed, the uncertainty of future technological development adds to the difficulty of technology assessment. There are usually several options that can be used to attain the targeted functional performance or productivity. For example, chemical methods as well as biological methods can be used in producing chemical compounds; depending on the specific compounds, different methods are adopted. In a survey by the Department of Defense (the Critical Technologies Plan), optic technology is named as a critically important technology. Several performance targets are listed, and an evaluation of the technological levels of various countries is performed. However, some experts point out that the performance of certain targeted devices may be achieved by optical or nonoptical means; it is likely that many electronic devices will be used complementarily to an optical system.
Finally, careful consideration should be paid to which specific technologies or products are chosen as subjects of an assessment. To assess national technological capabilities, usually a group of selected high technologies is examined. However, high technology products account for only one part of the total industrial output. Traditional industries such as steel, petrochemical products, and automobiles hold larger shares of the total output than computers and aerospace-related products. Yet, one may argue that high technologies stimulate economic activities as a whole and are valid to measure in this respect. It is true that there are many high technology elements adopted in the production process of traditional industrial products. However, high technology will not bring a large benefit to a nation if its application is not pursued aggressively in those industries that have a large output. In other words, assessment of the technological capability of the industries that produce high technology commodities is not sufficient to evaluate the technological capability or competitiveness of all of a nation's industries.
OUTLINE OF THE TECHNOLOGY ASSESSMENT PROJECT CONDUCTED BY MITI FOR THE WHITE PAPER ON INDUSTRIAL TECHNOLOGY (1988)
The Ministry of International Trade and Industry conducted a technology assessment project covering 40 industrial products and technologies for the preparation of a white paper (published in 1988) on industrial technology. The purpose of this assessment was to identify Japan's strengths and weaknesses in specific technologies, at specific stages of research and development. It was supposed to be a trial that would illustrate complex realities of the method by assessing several different elements of technologies.
Technological levels were investigated in three areas, namely, products, key technologies, and research and development potential (Figure 2). The
technological level of products was assessed in three aspects: product performance, reliability, and price competitiveness. Similarly, key technologies were chosen with three criteria in mind: the capability of research, development and design, productivity and the ability of application. The research and development potential was evaluated from five different standpoints: the abilities in basic research, in strategic basic research, and in applied research; originality; and capability for improvement.
The survey produced several interesting findings. First, Japan had an advantage over the United States in reliability of many of the products. In product performance and in price competitiveness, Japan had an advantage for some products but was at a disadvantage regarding others; the overall performance of Japan was equal to that of the United States in parts, finished goods, and systems. In materials, the United States had an advantage in price competitiveness in six out of eight areas of technology. The United States led in many of the key technology areas, particularly in applications. The United States showed strength in pure basic research, strategic basic research, and the originality of R&D potential. Regarding the improvement factor, Japan's performance exceeded that of the United States. Unfortu-
nately, the details of the survey were not presented in the white paper. Instead, an aggregated performance was shown in the paper, in which the overall technological level and the R&D potential of key technologies were evaluated as a simple average of the performance in different aspects of products and R&D. Vital information regarding the realities of the various assessments was lost in the process of aggregation.
OBJECTIVES OF THE ONGOING SURVEY FOR THE PREPARATION OF THE NEXT WHITE PAPER ON INDUSTRIAL TECHNOLOGY
As discussed in this paper, the realities of technological assessment are very complicated and a simple comparison of overall technological performance does not do the issue justice—and, therefore, can be misleading. This was an important lesson learned by MITI through the last survey. The natural question, then, is what can we learn from technology assessment, and how should we go about doing so?
The experience of the previous survey provided a hint. Technology assessment can explain the reality of technology: that is, what technology is, and how it is generated and used. In other words, technology assessment can be used as a tool to identify how scientific and technological activities of different disciplines interact, how one industry is dependent on others in terms of technology, to what extent the globalization of scientific and tech-
nological activities prevails, how innovation takes place, and what the environmental factors that stimulate innovation and diffusion of technology are. Indeed, these are the objectives of the ongoing survey (on technology assessment) that is being conducted by MITI for the preparation of the next white paper on industrial technology, set to be published next spring.
In the survey, some 20 industrial products will be examined. They will be chosen not for their technological sophistication, but by the size of the portion of the market that they hold as an objective criterion. Therefore, many of them will not be high technology products; for example, offset printing and the automobile industry will be highlighted.
The survey intends to evaluate the products (industries) from four as-
pects: product performance, performance of the production processes, research and development potential, and the environment for R&D. In each of the areas, as many indicators as possible will be collected in the hope of an objective analysis. The set of such indicators will be different for different products. Any important technology used in the production of the products, regardless of whether it is new or old, will be cited for consideration in the evaluation. The survey is expected to explain the evolution of certain technologies, as well as the conditions and environments that facilitated or inhibited the development of these technologies.
Also efforts will be made to identify these technologies as original, licensed domestically, or imported. The reason for such efforts is not so much that MITI wants to compare Japan's original technologies against foreign technologies, but because of the intent to illustrate the interdependence between firms and industries on a global level.
In this respect, the previous survey also tried to identify the extent of interdependence (see Figure 3). It showed considerable interdependence in
all of the 40 products surveyed. However, we should be careful in reading the results. The results were based on the aggregated data of how experts saw the situation in their own technological fields, including consideration of both qualitative and quantitative aspects. It does not necessarily follow that some specific technology, though a small share, made an unimportant contribution to production. The matter of concern is that however small the contribution may have been in percentage terms, the product would not be produced without the technology. The new ongoing survey puts focus on this interdependence.
It is essential that policymakers understand the conditions and environments for innovation and diffusion of technologies, the situation of deepened interaction between scientific and technological activities of different disciplines, and the globalization of these activities. The reason for this is that a suitable policy is one that would provide appropriate conditions and environments for technological development. This was recommended in the Organization for Economic Cooperation and Development (OECD) Ministerial Council last June as a result of three years of discussion under a special program entitled the "Technology and Economy Program." In addition, the globalization of scientific and technological activities has illuminated the issue of bringing domestic policies to a point of international harmonization. For this purpose, the OECD will discuss whether there is a need to develop additional "rules of the game" in the future.
Technology assessment will provide important information to policymakers on how technology evolves and is disseminated. In other words, it will tell them details about the conditions and environments that stimulate innovation and the diffusion of technology. Therefore, policymakers should be better equipped to develop policies that would create favorable conditions for innovation and the diffusion of technology.
Another important message that can be transmitted through technology assessment is the reality of deepening interdependence in industrial activities beyond natural boundaries. The question that should be raised is not to what extent should we depend on foreign sources for the supply of critical commodities and technologies, but rather how can we sustain stable relations between countries so that we can maximize the benefits of interdependence?