National Academies Press: OpenBook

Prospectus for National Knowledge Assessment (1996)

Chapter: 2 How to Do a Knowledge Assessment

« Previous: 1 What is a National Knowledge Assessment?
Suggested Citation:"2 How to Do a Knowledge Assessment." National Research Council. 1996. Prospectus for National Knowledge Assessment. Washington, DC: The National Academies Press. doi: 10.17226/9528.


How to Do a Knowledge Assessment

The implementation of the knowledge assessment methodology will be a collaborative effort among the World Bank, an independent implementing team recruited by the Bank, and local participants. World Bank staff stationed in the country will be counted upon to provide a briefing for participants on the economic and policy status of the country. In addition, individual members of the staff may serve as expert members of the team. The implementation team will not be fully formed until after the national symposium at which the sentinel enterprises are selected, because the team will include experts familiar with such enterprises and have access to global databases to establish appropriate benchmarks.

The local members of the team will include experts drawn from the academic and the business community. In the event that other sectors such as health, education, and agriculture are emphasized in the knowledge assessment, then experts from those areas must also be included. Inclusion of members drawn from those government agencies or ministries that have responsibilities in the areas relevant to the knowledge assessment merits careful, case-by-case consideration. Thus, representation from the ministries of industry, education, telecommunications, information, and trade should be based on compelling reasons to include certain individuals. The principle to be respected is to avoid confrontation and enhance cooperation. If it can be argued that the presence of entrepreneurs and ministry people at the same table will produce an alliance among them, their presence should be sought. If the possibility of confrontation would stifle frank discussion and exclusion would not bias government officials against the study, then it should be minimized.

For somewhat different reasons, it is recommended that World Bank staff participate but the Bank not carry out the knowledge assessment itself. As the premier development organization in the world, the World Bank has become associated with certain positions on issues that are relevant to the knowledge assessment. Questions of deregulation, protection, educational priorities, and others where the Bank has a well-known position may be subjects of recommendations arising from the knowledge assessment, and the control of the Bank, or its dominating presence on the team, may lessen the impact of the recommendations. Since the product of the knowledge assessment is purely advisory, Bank staff would have ample opportunity to challenge the recommended options in the future, leading the participants to refine and strengthen their arguments.

The Bank briefing for the implementation team will become an important part of the record of the knowledge assessment. It should cover the state of the economy, the infrastructure, macroeconomic policies of the government, human resources, and other topics that the Bank mission considers relevant.


Three important tasks must be carried out before the various teams assemble to carry out the knowledge assessment. These are:

  • Identify local stakeholders from the private and public sector who will participate actively in the knowledge assessment and, as a result of their participation, generate the momentum and constituency for change.

  • Select the sentinel enterprises that will illuminate the flows and the mechanisms for and extent of application of knowledge in the country.

  • Prepare a map of the National Knowledge System, providing in effect an organogram and flow diagram that can be used to pinpoint the barriers and high leverage points that will be the focus of the recommendations.

These three tasks must be well prepared in advance by the assessment team before they can be accomplished at a National Symposium. The symposium, possibly organized by the World Bank, will bring together entrepreneurs in knowledge-related industries, scientists and engineers, and government officials with responsibilities in the information or knowledge area, as appropriate. Members of the implementation team should lead the symposium to make certain that

Suggested Citation:"2 How to Do a Knowledge Assessment." National Research Council. 1996. Prospectus for National Knowledge Assessment. Washington, DC: The National Academies Press. doi: 10.17226/9528.

participants get a clear idea of the purpose and process of the knowledge assessment. The participants will include those most knowledgeable in the country of the workings of the institutions that make up the National Knowledge System, and their contribution will be essential.


Preparation for elaboration of the National Knowledge System will begin at the national symposium. The starting point will be a template of the functions normally associated with a national knowledge system. This template, upon which the map will be based, must have been already developed from the pilot project described below and the accumulated experience of earlier knowledge assessments in other countries. A few examples of the type of functions that appear in the template, organized according to motivation, creation, access, assimilation, diffusion, and use, are given in table 1 . It will be the first task of the national symposium to organize and label these functions according to the agency, ministry, or private organizations that have responsibilities to perform them in the country. These will include the universities, government agencies, professional and industrial associations, private firms, non-government organizations, and others. The labeling can be carried out by the symposium participants in working groups, and the discussion and analysis of which agencies have which responsibilities will have a fruitful impact on later discussions of barriers.

Once the labels have been put in place, the flow of knowledge will be charted. Beginning with the flow of knowledge into the countries, it will include the sources of knowledge within the country and the knowledge users, and it will show the links among the agencies mapped into the National Knowledge System.


A virtual case study is similar to preparing a feasibility study for a new enterprise or the expansion into a new process, product, or service by an existing one. All the considerations that pertain to a real enterprise are reviewed step by step for the hypothetical enterprise. However the objective is not to plan a real investment but to identify barriers that prevent such enterprises from coming into existence or that make them unattractive, or to identify new mechanisms that might facilitate them.

The success of the virtual case study depends greatly on the selection of the participants, their knowledge of the country and of the industry or sector that embraces the sentinel enterprise, their familiarity with economic, cultural, and intellectual conditions in the country, and the skill of the facilitators. An unknown in each country is the cultural factor that may or may not make it comfortable for participants to join in this kind of “game.” (An experiment in structured group brainstorming carried out at the Organization of American States some years ago was pronounced by some unhappy participants “very gringo.”) In some countries local participants may gladly volunteer for this service, realizing that it is in their best interest or the best interest of their country. In others, they may demand to be paid as consultants. Many businessmen and women may see it as a risky activity from which they may personally lose or gain face or advantage. In many countries, it will be best to carry out the virtual case studies in English; in others the local language will facilitate discussion. Experience, beginning with the pilot project, will determine how to maximize the yield from this exercise in each culture.

Selection of Sentinel Enterprises

The selection of the sentinel enterprises will normally take place at the national symposium, where local stakeholders and World Bank staff will take an active role. Two issues are most important. The aim is not to “pick winners” in economic competition, it is to eliminate barriers so that winners will be able to flourish. Therefore it is essential that the sentinel enterprises be selected with due regard to the national economic reality and for the priorities set forth in national development plans.

With this in mind, the sentinel enterprises are intended to expose the weaknesses of the National Knowledge System and the barriers to productive use of knowledge. Not all the barriers will be directly related to knowledge and information; high fees or other impediments to establishing enterprises of any sort, high import or export duties, or poor airport facilities would be detrimental to a knowledge-based enterprise as it would be to any other. Yet a judicious selection of sentinel enterprises will point a finger at the same key set of barriers from different perspectives, and provide both the rationale and the basis for evaluation of recommendations for removing such barriers.

If the industrial sector is the target of the knowledge assessment, then a sample of sentinel enterprises might include a sophisticated financial service company, a venture capital company, a computer software company, an engineering consulting company, a precision foundry, a plant to process a natural product, a food product company based on a local formula, and a company to publish technical books and journals. It must be stressed again that the aim is not to lay the groundwork for establishing such companies nor to encourage others to compete with existing companies. The only market research that will be done is to determine whether market research can be done, given the available knowledge resources. Similarly, the recruitment of staff and managers will be limited to determining whether appropriate staff and managers might be recruited in the country or from abroad. Any participant who is encouraged by the proceedings to start such an enterprise will be entirely on his or her own. A participant who sees the threat of competition from analysis

Suggested Citation:"2 How to Do a Knowledge Assessment." National Research Council. 1996. Prospectus for National Knowledge Assessment. Washington, DC: The National Academies Press. doi: 10.17226/9528.

TABLE 1 Functions of a National Knowledge System

Motivation Functions:

Formulation, monitoring, and review of policies and plans affecting national knowledge activities;

Allocation of resources to science and technology;

Preparation of national development plan;

Establishment of laws and regulations affecting communications and trade;

Creation of incentive schemes to stimulate knowledge application and other technical activities;

Technological forecasting and assessing the likely directions of new knowledge and technical change;

Regulation of the stock market and other investment systems;

Use of government's purchasing power as a stimulus to knowledge-based enterprises;

Stimulation of public interest in and support of learning and innovation;

Creation Functions:

Formulate policy and set priorities for research and development funding;

Carry out basic and applied research and development;

Operate major national facilities for research and scientific training;

Provide awards and recognition for successful original research;

Monitor and evaluate the quality of research and development;

Access Functions:

Creation and operation of a national system for the recording and protection of intellectual property;

Provision of linkages to regional and international information sources;

Establishment, operation and maintenance of information services (including libraries, data bases, statistical services, system of indicators, communications systems);

Provide up-to-date knowledge on advances in capital goods and process technologies;

Provide opportunities for foreign study for public and private sector employees;

Publish statistical indicators on science and technology;

Publish register of laws and regulations;

Assimilation Functions:

Establish curricula and standards for science and engineering education;

Provide programs and facilities for vocational and on-the-job training;

Train engineers and researchers;

Operate technological observatories abroad to advise firms on opportunities and threats;

Diffusion Functions:

Maintain a national system for metrology, standardization, calibration, information, and statistics;

Operate technical services (e.g. metrology, standardization, calibration, information, statistics);

Publish technical journals and newsletters;

Organize scientific symposia and seminars;

Provide technical consulting services;

Evaluate and diffuse best practice technologies;

Use Functions:

Provide assistance in negotiation for technology transfer, licensing, and joint ventures;

Promote innovative goods, processes and services embodying knowledge;

Carry out export market research for local firms;

Provide technical consulting service on quality control;

Provide quality testing service accredited in major foreign markets;

Provide advice to firms on opportunities related to GATT and other international marketing agreements.

Suggested Citation:"2 How to Do a Knowledge Assessment." National Research Council. 1996. Prospectus for National Knowledge Assessment. Washington, DC: The National Academies Press. doi: 10.17226/9528.

of a sentinel enterprise that is close to her own business must realize that she herself would be in the best position to take advantage of the presumed opportunity, but she must be encouraged to share her knowledge in order to benefit from the knowledge of the other experts.

Sample Sentinel Enterprise

For the sake of example, we might imagine one sentinel enterprise to be a software engineering firm. This type of firm would enable other firms to obtain new software to enhance their access, speed, agility, and quality management for a global market. In particular it might be able to produce sophisticated software in the local language that would increase its acceptability and ease of use for local clients.

The following types of individuals might participate in the virtual case study:

  • a software engineer from an international computer company

  • a computer or software engineer from a local company

  • a major software user

  • a professor of computer science at the national university

  • a local computer dealer

  • a representative of an industry group or association, users of computers

  • a representative of the finance community

  • a local patent or regulatory lawyer

  • members of the knowledge assessment team, with previous experience of virtual case studies.

  • an experienced facilitator.

The issues raised for discussion will include:

  • preparedness of persons or firms for starting such an enterprise

  • access to capital

  • availability of communications infrastructure

  • access to market information

  • consumer receptiveness to product or service

  • right to lease premises

  • procedures and permits required to launch a new enterprise

  • technical assistance on business practices

  • access to information on business software

  • technical assistance for selection of equipment

  • right to import equipment and software

  • availability of skilled labor

  • training opportunities

  • information on best practices

  • environmental protection, as appropriate

  • information and training on quality management

  • regulation on health and safety

  • opportunities for joint ventures, and permits required to launch them.

The exercise should be carried out in about a day.

If well chosen, the issues raised here will be similar to those for the next sentinel enterprise selected, which might be a formula-based soft drink company. The selection of two or more concrete but very different industries allows different elements of the industrial sector to become engaged and reinforces the generality of the recommendations.

It is essential that the participants be open and frank. Barriers will be recognized only when statements like: “that agency is very inefficient” or “bribes must be paid to complete that transaction” are heard and noted. Some of the issues will not be knowledge-related, but where knowledge is not the limiting factor, other types of barriers must be removed before knowledge can be effective.

Baseline and Benchmarks

For each sentinel enterprise, if similar enterprises already exist in the country, baseline data will be gathered to form a basis for later evaluation. This information will be gathered partly from the local participants in the virtual case studies, and partly from the data and interviews. It will include average and range of firm size, capitalization, technology used, health and safety standards, quality measures, skill level of employees, and market size.

Benchmarks are provided by the international participants. These are based on information obtainable from international, industry-wide data bases containing industry averages and best practices, indexed by country. The particular metrics used should correspond to the baseline data, if possible, to facilitate direct comparison for evaluation purposes.

Here is a rather simple example. One of the functions in the template might be to license the importation of equipment. The agency responsible is, say, the Customs Bureau, and this appears on the National Knowledge System map. Links in one direction are to the Ministry of Finance, and in the other to the industrial sector. For the case imagined above of a software engineering company, the “equipment” might be a sophisticated software package used for design of applications. The virtual case study will show that the relevant metrics are the time required to obtain the import license and the fees involved. The baseline is the current time and fee in the country. The best practice might be the time and fees charged in Singapore, or the maximum that a global firm such as IBM or AT&T could tolerate to consider a contract for design of software with such a company.


The virtual case studies will be an effective tool for probing the National Knowledge System, but at best they must be treated as revealing, albeit fictitious, anecdotes. The issues raised must be confirmed by valid data before recommendations can be made. For this reason, the data collection and interviews should be carried out after the virtual case studies

Suggested Citation:"2 How to Do a Knowledge Assessment." National Research Council. 1996. Prospectus for National Knowledge Assessment. Washington, DC: The National Academies Press. doi: 10.17226/9528.

are completed. The barriers identified by the virtual case studies will guide the selection of statistical data and provide the issues for the interview guides. Specifically, the barriers should be raised in the interviews in order to confirm the conjectures of the case study teams and to gather ideas for lifting barriers, building bridges, and remedial action.

The following quantitative indicators are intended to provide an example of knowledge available in the country and in international databases that will be relevant and available. Similarly the issues suggested below could be included in the discussions with individual entrepreneurs, scientists and engineers, government officials, and other knowledge workers. The pilot project will produce a more complete list of indicators with general sources and interview guides based on the virtual case studies.

Sample indicators and issues for the interview guides are listed according to the element of the National Knowledge System.


The use of knowledge for productive development depends on the motivation and example provided by the leadership of the country, the government, and the productive sectors, and the incentives, financial and otherwise, for its application. Also included are enabling conditions like physical infrastructure and laws and regulations.

Sample indicators

financial structure

taxes and duties

cost of capital

cost of labor

number of exporting firms

venture capital sources and availability

limits on repatriation of earnings






postal services

laws and regulations

local content requirements

laws governing foreign direct investment

laws governing importation of technology

Sample interview issues


political stability

economic stability

trust in institutions

pent-up demand

political climate toward innovation


Creation of knowledge is the essence of the research and development system. However in most developing countries scientific resources and manpower are limited and not competitive with the best laboratories in the advanced countries, except on purely local scientific issues. Therefore institutions dedicated to the creation of knowledge should be linked to those providing access to knowledge for best allocation of resources.

Sample indicators

input indicators

number, quality of research facilities

number, quality of research graduate programs

R&D budget, by field and institution

output indicators

published papers; patents

collaboration with foreign scientists and engineers

Sample interview issues

rank in field of key research groups

contribution of research and development to productivity

adequacy of resources and manpower

linkage between university labs and industry


Motivation to engage in knowledge-based productive activity will be in vain if there are not sources of information and of knowledge available within the country. The interesting indicators will relate to the institutions that provide the knowledge. These will describe sources of both embodied and codified knowledge, because their knowledge stock must be measured in learned people as well as books and products.

Sample indicators


number, quality of research universities

information services

existence of data bases

numbers of journals, newsletters, seminars

global networking

number of Internet users, servers

companies using Internet

on-line information services

PCs per capita

telephones or telephone lines per capita

membership in professional societies

numbers of joint ventures or amount of direct foreign investment

movement of peoples

Suggested Citation:"2 How to Do a Knowledge Assessment." National Research Council. 1996. Prospectus for National Knowledge Assessment. Washington, DC: The National Academies Press. doi: 10.17226/9528.
Sample interview issues

political openness

access to government information sources

free press


legal structure relating to knowledge


Information is a tool, one that takes a trained worker to use effectively. The workers must know how to use the tool, and they must have it available precisely when it is needed. Information and knowledge may be accessible in a country, but they must be understood to be used. The question of assimilation involves human capacity, the availability of skilled and motivated people.

Sample indicators

public and private expenditures on R&D

workforce characteristics

natural scientists and engineers

technicians (bachelors and two-year college)

where graduates are employed

educational structure

literacy rate

fraction of students that attain various educational level, males and females

graduates in selected fields

foreign graduates, profile and return rate

number of research universities

computer education

continuing education; training programs

education of managers

Sample interview issues

business environment

preparedness of university graduates for industry

influence of Internet and other networks

antitrust policies.


Diffusion will depend upon communication links within each sector, and between these and the universities and research laboratories. Mobility of workers and existence of trade and technical publications contribute to the diffusion of knowledge.

Sample indicators

mobility of labor and management

number and type of industrial associations

number and type of knowledge parks and other sharing facilities

benefit plans that encourage or discourage hijacking of trained labor

Sample interview issues

inter-firm linkages

“communicativeness” and networking

attitudes toward collaboration

role of associations in spread of knowledge

use of trade shows

influence of capital goods industry

other mechanisms that spread knowledge


Use is the concrete result of motivation, access, and assimilation and diffusion of knowledge. Data gathered in this regard will show the success or failure rate of knowledge-based enterprises.

Sample indicators

industry structure

patent and licensing data

number of knowledge-based enterprises

size distribution

local and foreign

imports and exports of technological goods

Sample interview issues

ease of establishing knowledge-based enterprises

consumer acceptance of innovation and new products

factors contributing to success or failure of knowledgebased enterprises

experience of best cases

Statistical Sources

Quantitative information will come largely from national sources, government agencies or private associations. Some international agencies have data for certain groups of countries.

Bureau of statistics

industrial surveys, educational surveys

Science and technology indicator reports


Industry associations

Interview Sources

The qualitative information will come from public officials, scientists and engineers, business men and women, and other producers. The selection should be focused around the

Suggested Citation:"2 How to Do a Knowledge Assessment." National Research Council. 1996. Prospectus for National Knowledge Assessment. Washington, DC: The National Academies Press. doi: 10.17226/9528.

institutions identified as playing a role in the National Knowledge System.

Political leadership

Social and religious leadership

Information industry

Health providers


Chamber of commerce

Industry associations

Engineering associations

Professional associations

CEOs, large, global, and small and medium enterprises


Recommendations will be aimed at establishing sustainable ways to maintain development. That is, permanently removing barriers related to knowledge or instituting incentives for productive use of knowledge will allow a continuous process of establishing knowledge-based enterprises, competition, incremental innovations, and continuing social and economic development.

Following the themes of the knowledge assessment, recommendations will consist of positive steps to improve motivation, creation, access, assimilation, diffusion, and use. Some of the recommendations may be more general, identifying barriers that are not specific to knowledge-based enterprises, such as regulation, energy, and infrastructure. These must be included, because it may be that knowledge-related motivation, access, assimilation, and diffusion are not the limiting factors, and one can not expect to see a change if the other issues are not dealt with.

Recommendations should arise from knowledge assessment in a systematic way. The elaboration of the National Knowledge System, based on the template developed during the pilot project, will reveal institutional weaknesses on a coarse-grained scale. Lack of a government office to carry out an important function identified on the template will alert the team to a possible recommendation to remedy that lack. A published register of patents and trademarks, an overseas technology observatory linked to the embassies, or an information service for industry with access to international sources are examples of candidates for such recommendations.

The source of more specific recommendations will be the virtual case studies. These will identify not just gross features like absence of an office or service, but weaknesses in laws and inefficiency in services that do exist on the National Knowledge System map. It is also here that the non-knowledge specific recommendations will arise, when participants see obstacles to success of the sentinel enterprises that are part of broader problems. But the greatest value will come from the identification of specific inefficiencies: shortage of entrepreneurs or managers, management that does not recognize its key leadership role in encouraging innovation, regulations that are not published, agencies that are corrupt or understaffed, services that do not serve, universities whose graduates are unprepared, or leadership that is unaware of the importance of knowledge and technology. When these are superimposed on the National Knowledge System map, that data that must be collected and the officials and knowledge users who must be interviewed will become apparent, and a program of data collection and interviews will be carried out to reinforce and quantify the recommendations. Based on all these inputs, the recommendations, in the form of a program of remediation, bridge-building, and implementation, may be designed to stimulate sustainable knowledge-based development in the country.

The recommendations and an implementation plan will be prepared by the knowledge assessment team, supplemented by local experts drawn from the virtual case study teams. The global experts, familiar with benchmarks, and the local experts, familiar with local conditions, will provide the balance of opportunity and realism that characterizes a workable plan.

Suggested Citation:"2 How to Do a Knowledge Assessment." National Research Council. 1996. Prospectus for National Knowledge Assessment. Washington, DC: The National Academies Press. doi: 10.17226/9528.
Page 16
Suggested Citation:"2 How to Do a Knowledge Assessment." National Research Council. 1996. Prospectus for National Knowledge Assessment. Washington, DC: The National Academies Press. doi: 10.17226/9528.
Page 17
Suggested Citation:"2 How to Do a Knowledge Assessment." National Research Council. 1996. Prospectus for National Knowledge Assessment. Washington, DC: The National Academies Press. doi: 10.17226/9528.
Page 18
Suggested Citation:"2 How to Do a Knowledge Assessment." National Research Council. 1996. Prospectus for National Knowledge Assessment. Washington, DC: The National Academies Press. doi: 10.17226/9528.
Page 19
Suggested Citation:"2 How to Do a Knowledge Assessment." National Research Council. 1996. Prospectus for National Knowledge Assessment. Washington, DC: The National Academies Press. doi: 10.17226/9528.
Page 20
Suggested Citation:"2 How to Do a Knowledge Assessment." National Research Council. 1996. Prospectus for National Knowledge Assessment. Washington, DC: The National Academies Press. doi: 10.17226/9528.
Page 21
Suggested Citation:"2 How to Do a Knowledge Assessment." National Research Council. 1996. Prospectus for National Knowledge Assessment. Washington, DC: The National Academies Press. doi: 10.17226/9528.
Page 22
Next: 3 Next Steps »
Prospectus for National Knowledge Assessment Get This Book
MyNAP members save 10% online.
Login or Register to save!
  1. ×

    Welcome to OpenBook!

    You're looking at OpenBook,'s online reading room since 1999. Based on feedback from you, our users, we've made some improvements that make it easier than ever to read thousands of publications on our website.

    Do you want to take a quick tour of the OpenBook's features?

    No Thanks Take a Tour »
  2. ×

    Show this book's table of contents, where you can jump to any chapter by name.

    « Back Next »
  3. ×

    ...or use these buttons to go back to the previous chapter or skip to the next one.

    « Back Next »
  4. ×

    Jump up to the previous page or down to the next one. Also, you can type in a page number and press Enter to go directly to that page in the book.

    « Back Next »
  5. ×

    Switch between the Original Pages, where you can read the report as it appeared in print, and Text Pages for the web version, where you can highlight and search the text.

    « Back Next »
  6. ×

    To search the entire text of this book, type in your search term here and press Enter.

    « Back Next »
  7. ×

    Share a link to this book page on your preferred social network or via email.

    « Back Next »
  8. ×

    View our suggested citation for this chapter.

    « Back Next »
  9. ×

    Ready to take your reading offline? Click here to buy this book in print or download it as a free PDF, if available.

    « Back Next »
Stay Connected!