Internet Counts: Measuring the Impacts of the Internet (1998)

This chapter describes the types of impacts and indicators with which this report is concerned. The importance of recognizing and anticipating positive as well as negative impacts of the Internet is noted, and the differences between direct and indirect, and intended and unintended, impacts are presented. The discussion then proceeds to the limitations and qualifications of indicators in and of themselves and as used in this report. For the subject at hand, the committee found that the most useful method of looking at impacts of the Internet is to begin with the most direct and move to the more indirect. Finally, the chapter notes the importance of using qualitative measurements in addition to quantitative indicators to develop enough understanding of the processes under way to guide the development of appropriate indicators.

In this report, direct impacts are those that are an immediate consequence of the introduction or use of the Internet. Introduction of the Internet in Africa and other countries has some direct impacts that are relatively easy to understand and measure, such as the reduction in communications costs for some uses. One can easily identify and quantify the reduced costs of sending e-mail versus a fax. There are other direct impacts that are easily identified, but somewhat less easily quantified such as the increasing ease of some communications and improved timeliness of communications.

Still other impacts are less obvious and more difficult to measure but perhaps more important in the long run. For example, how do we assess the value of an individual or organization increasing international contacts and participating in activities that previously were unknown or out of reach? In this report, indirect impacts are those that are not an immediate and exclusive result of the Internet. They may depend on who communicates what to whom and why.

Indirect impacts may also depend on changes that occur in programs and institutions as a result of the technological opportunities made available by the Internet. Indirect impacts can range from increased cost effectiveness of a small nongovernmental organization (NGO) to restructuring of an economy as a result of changes in the structures of comparative advantage induced by the Internet.

It is unlikely that all impacts of the Internet are or will be positive. Using television as a comparable technological innovation, some studies have concluded that TV is responsible for making violence more acceptable in U.S. society, for increased emphasis on consumerism, and for reducing literacy by substituting television viewing for reading as a pastime for large numbers of people. There is also growing concern about criminal activities, dubbed “cybercrime” by some, on the Internet. Any complete analytical framework for the Internet must provide for both positive and negative impacts. Moreover, any single stakeholder, be it an individual Internet user, development agency, or policymaker, will likely see both positive and negative impacts from Internet use and will have to balance those impacts according to his or her individual needs and expectations.

In addition to positive and negative impacts, one can distinguish between intended and unintended impacts of introduction and use of the Internet. These can be either positive or negative and depend on the perspective of the analysis. For example, decreased international communications costs, and hence decreased costs of sending data to another country, may be the positive impact an NGO desires upon acquiring Internet access; but when that NGO outsources analytical work that was previously done inhouse and incountry to analysts in another country, it is an unintended and negative impact from the perspective of a development agency striving to increase local employment and Internet experience.

Impacts of the Internet occur at several levels, including the individual, organizational, and sectoral levels and, as noted earlier, the impacts may be positive at some levels or for some stakeholders and negative at other levels. An individual user of the Internet will feel personal impacts, in terms of greater access to information and people, the cost of acquiring information, time budgets, and so forth. But the cumulative impact on an organization is probably greater than the sum of the impacts on individual users. That is, an organization will likely change structure and process to take advantage of the new opportunities provided to individuals by the Internet. Similarly, the impact on an industry may not be simply the sum of the impacts felt by the companies in the industry, where the industry restructures and modifies its processes in light of the opportunities and risks it perceives. Governments and donor agencies are also interested in the impacts of introduction of the Internet on society as a whole. Thus, an analysis of the cumulative impact is difficult unless one is able to analyze individual impacts together and look for the resulting balance.

The approach taken below deals with indicators of the penetration and use of the Internet and indicators of its impacts at the level of various institutions. On the basis of their studies of West Africa, the committee members suggest that an appropriate strategy to measuring the complex and indirect impacts of the Internet in countries where it is not widely used would be to trace the impacts from the most direct to the increasingly indirect. Thus, while many changes within organizations will take much time to be visible and measurable, the impact on organizations can be partly estimated from knowledge of the impacts on the users and nonusers of the Internet in the organization, as well as changes in the structure and processes of the organization made as a result of introduction of the Internet. Similarly, the impact on a market can be partly inferred from the impacts on the companies that buy and sell in that market, the organizations that regulate or operate the market, and changes made in the structure and processes of the market based on introduction of the Internet.

Development agencies and others typically use indicators to define how performance or impacts will be measured along a scale or dimension, without specifying a particular level of achievement. Such indicators are an increasingly important management tool.

Performance indicators are at the heart of a performance monitoring system–they define the data to be collected to measure progress and enable actual results achieved over time to be compared with planned results. Thus, they are an indispensable management tool for making performance-based decisions about program strategies and activities. ⁷

There are a number of useful classifications of indicators. For example, below we differentiate between quantitative and qualitative indicators. The following discussion includes quantitative indicators, such as the number of Internet services provided, and qualitative indicators, such as the quality of the services, as well as quantitative indicators on the amount of content available on the Internet and qualitative indicators of the quality of that content.

While quantitative indicators are not necessarily more objective, their numerical value tends to lead to more agreement on interpretation of results data, and they are often preferred by program managers. However, qualitative indicators can supplement the numbers and percentages and, thereby, add a real-life perspective to the evaluation of a program.⁸ Indeed, it will hardly be possible to find quantitative indicators for some of the indirect impacts of the introduction of the Internet in large and complex socioeconomic systems, and qualitative indicators may be the only ones available that are relevant to key policy and management issues. A further challenge is to devise a model to explain the qualitative indicators and their significance.

Indicators that measure numerical values are commonly divided into such groups as ratios, intervals, ordinal values, and nominal values. The division is important in terms of the analysis that the indicators will support. Another classification is input, output, and outcome indicators. In the following text, many indicators are provided that measure ratios, such as the percentage of information and data an institute obtains via the Internet and the percentage of business organizations with Internet access. Also suggested are ordinal indicators such as whether users find electronic mail to be of no use, marginally useful, useful, very useful, or indispensable.

There are a number of limitations and qualifications that must be borne in mind when considering and using the impact indicators presented below. Given the wide variety of direct and indirect impacts of introduction of the Internet, there will be a large number of potential indicators related to such impacts. An immediate concern is simplicity, as busy people could be overwhelmed by the demand for data and overworked analysts could be overwhelmed by an excess supply of data. Moreover, data beyond very basic supply and use statistics may be considered confidential or be otherwise hard to obtain.

The indicators are intended to provide a framework for local and foreign users, providers, and policymakers for collecting, organizing, and analyzing information. The indicators are intended as guidance for people who will choose areas and indicators to meet their own needs: a development agency evaluating its Internet project in an urban setting will use different indicators than will a local education minister evaluating the costs and benefits of investing in computers for Internet access in a rural school. Appropriate subsets of the indicators could also be used for large cross-national studies that allow comparison of two or more countries, provided adequate explanation is given as to differences between the countries in all factors related to the Internet.

Indicators and the analyses based on them are decision aids. The value of indicators stems from the analytical purposes for which they are used, and ultimately from the improvements in

policy and management to which they contribute. Some analyses of impacts of the Internet in Africa will require complementary indicators that are not addressed in detail in this report, such as social indicators for measuring economic well-being and development. They are not addressed here because there is a huge and excellent literature on the subject, and that literature is too extensive and profound to summarize adequately in this report. However, instruments that collect data on indicators of Internet penetration, use, and impacts must also collect corresponding data needed for proper analysis of the impacts. For example, based on the committee's research in Senegal and Ghana, it is clear that males have greater access to the Internet and use it more extensively worldwide than do females. Gender information will be required to consider equity issues. Comparably, geographic indicators will be required to differentiate urban from rural impacts as well impacts by region of a country. Information on language of use and ethnicity of users will similarly be needed for many analytical purposes.⁹

Another concern is the ability to sustain causal attribution through lengthening chains of indirect impacts. While causality is implied by the term impact, the key analytical problem in application of the indicators will be attribution of the changes observed in the impact indicators to introduction of the Internet. Thus, for example, the Internet may contribute to restructuring of the economy in some advanced countries, but it is not possible to definitively identify how much of the observed change is due to the Internet and how much is due to other factors.

There is also a need to look at those individuals and institutions in Africa who are not using the Internet and are likely to feel certain impacts or repercussions as a result. Nonusers may be affected indirectly by the uses others make of the Internet. For example, a farmer in Ghana may find a larger market for his produce if the intermediaries who buy that produce and sell it to others successfully use the Internet to expand their output markets. Alternatively, if exporters from other countries are sufficiently more successful in using the Internet to market to Europe than those buying from the Ghanaian farmer, he may suffer the consequences in lower sales and less income. It is important that, in planning for the development of impact indicators and in analyzing the impacts of the Internet, the impacts of nonuse at all levels be considered as well.

Although the framework of this report and most of the indicators proposed below suggest emphasis on quantitative approaches to analysis of numerical data, the committee strongly endorses the need for use of ethnographic techniques and the analysis of qualitative information to understand more fully the impacts of greatest interest. However, a discussion of such analysis methods is beyond the scope of this report.¹⁰

There are, of course, many impacts of Internet use and impacts that the indicators presented in this report will not measure. As new uses and applications of Internet technologies are developed and as Internet use increases globally, its impacts will also increase and expand, sometimes faster than our ability to develop appropriate means of understanding and analyzing them. It is important to keep the limitations noted above in mind and to realize that any list and any measurements will be incomplete and likely undervalue the true impacts in any one country,

region, or institution. The limitations do not, however, invalidate the value of a framework for analysis.