Most people who use the Internet rely on the Domain Name System (DNS) and navigation aids and services to find the resources they seek or to attract users to the resources they provide. Yet, although they perform well, both the DNS and Internet navigation services face challenges arising from technological change and from institutions with a wide variety of commercial, cultural, social, and political agendas. Individually, or together, those pressures could force operational changes that would significantly reduce access to Internet-linked resources by segments of the user community, reducing the Internet’s value as a global resource.
This document reports the conclusions of an assessment of the current state and the future prospects of the DNS and its interactions with Internet navigation, including its uses as a means of navigation itself and as an infrastructure for navigation by other means. The assessment is the result of the deliberations of a committee that encompasses a wide range of disciplines, experience, and viewpoints. The report is addressed to the technologists, policy makers, and others whose decisions will affect the future of the DNS and Internet navigation aids and services. The specific conclusions and recommendations of the Committee on Internet Navigation and the Domain Name System appear throughout this summary in boldface type.
DOMAIN NAME SYSTEM
Domain names are commonly used to designate services and devices on the Internet, as a more memorable and more permanent alternative to
the numerical addresses employed by its routing computers. They are the valued, often valuable, and often user-friendly names on the signposts that designate many things connected to the Internet. Consequently, which names are available, who controls their allocation, what is charged for their use, how their uses are managed, and the answers to many related questions are important to virtually everyone who uses the Internet, whether as information seeker or provider.
Overall, the DNS’s technical system and institutional framework have performed reliably and effectively during the two decades of the DNS’s existence. The DNS has coped with the extremely rapid expansion of Internet usage driven by the wide deployment of the World Wide Web in the 1990s and the widespread adoption of e-mail. The hierarchical, distributed structure of the DNS technical system, operated collaboratively by a group of mostly autonomous organizations, has proven to be scalable, reliable, secure, and efficient.
The DNS technical system can continue to meet the needs of an expanding Internet. Early in the committee’s assessment it became apparent that it would not be fruitful to consider alternate naming systems. As noted, the DNS operates quite well for its intended purpose and has demonstrated its ability to scale with the growth of the Internet and to operate robustly in an open environment. Moreover, significantly increased functionality can be achieved though applications—such as navigation systems—built on the DNS, or offered independently, rather than through changing the DNS directly. Hence, the need did not seem to be to replace the DNS, but rather to maintain and incrementally improve it. Furthermore, given the rapidly increasing installed base and the corresponding heavy investments in the technical system and the institutional framework, the financial cost and operational disruption of replacing the DNS would be extremely high, if even possible at all.
However, the continued successful operation of the DNS is not assured; many forces, driven by a variety of factors, are challenging the DNS’s future. Required and desirable technologies to increase security and enable the use of non-Roman scripts for domain names are not being incorporated into the technical system as quickly as many would like. There are persistent and substantial controversies concerning the structure and policies of the DNS’s institutional framework. Moreover, there have been many efforts to use the DNS, because it exists and is so widely deployed, for many purposes for which it may not be appropriate. In addition, national legislation and court decisions are addressing Internet and domain name issues with potentially conflicting consequences for the operation of the DNS.
Like all public networked systems, the system of public domain name servers is threatened by a variety of purposeful attacks, both malicious and mischievous, by individuals or groups that aim to disable or divert their operations. The operators of the DNS are responding to these threats, but not all the desirable steps to ensure security have yet been implemented.
Denial-of-service attacks attempt to overwhelm key name servers and their links to the Internet with so much traffic that they are incapable of responding to legitimate queries. The root name servers have the capacity and capability to respond to many times the normal number of queries they receive, and have alternate connections to the network if some are blocked. Their ability to respond to attacks has been improved by some operators’ recent addition of multiple distributed copies (called “anycast” servers) of the base name servers, increasing both capacity and connectivity. In anticipation of future denial-of-service attacks and normal growth in demand, and to improve service globally, anycast server deployment should be expanded.
Notwithstanding the deployment of anycast servers and installation of backup servers at remote locations, the concentration of root name server facilities and personnel in the Washington, D.C., area and, to a lesser extent, in the Los Angeles area is a potential vulnerability. The need for further diversification of the location of root name servers and personnel should be carefully analyzed in the light of possible dangers, both natural and human in origin.
In response to the threat of alteration of messages being transmitted among name servers, the technical community has developed DNS Security Extensions (DNSSEC), which uses digital signatures to verify that the content of a message to or from a name server arrives unaltered and that its origin is as stated. DNSSEC only gives assurance that what was sent was not changed during transmission; it cannot and is not intended to assert that the message is factually correct. For example, DNSSEC has no
capability to guarantee that it is communicating the correct address for a given domain name. The security of the DNS would be significantly improved if DNSSEC were widely deployed among name servers for the root zone and top-level domains in particular, and throughout the DNS in general.
Although some steps have been taken, more could be done to continuously monitor the performance and traffic flows of the DNS so as to enable rapid detection of and response to attacks or outages.
The DNS works through the voluntary cooperation of its autonomous component entities. That cooperation, in turn, depends on their tacit agreement on two principles that together enable the Internet and the DNS to evolve and remain effective:
Universal open standards. The first principle is that the protocols and standards defining operation of the Internet and the DNS will be open and established by the Internet Engineering Task Force (IETF), an international voluntary organization of technical specialists. This technical framework enables every device on the Internet to connect to and communicate with every other, and it has been critical to the success of the Internet and the DNS. Because changes in Internet and DNS protocols, standards, and practices are matters of consequence beyond specific Internet services, alterations to the functions of or modifications to established standards and practices have traditionally been vetted by the IETF before being implemented.
Innovation at the edges. The second principle is that applications should be offered by devices on the edges of the Internet, rather than at the Internet’s internal nodes or on its links. In general, applications located at the edges have little effect on the stability of the Internet, so there is no need to regulate them. The DNS is not, strictly speaking, internal to the Internet (the translation service is performed by computers at the edges), but functions as though it were. It can thus be thought of as a core service, which although not absolutely necessary, is extremely useful in giving a relatively user-friendly face to Internet resources, and for enabling access to those resources even when their Internet addresses change. Moreover, it is a deeply embedded and ubiquitous service that enables other services and functions, including most aids to Internet navigation.
This tacit agreement governs the basic behavior of the many autonomous operators of the DNS, but there is also a need for an authority to make decisions about the allocation of limited resources central to DNS operations. The most critical of these decisions are the determination of which top-level domains (TLDs) shall appear in the root zone file of the DNS, which organizations shall be designated as responsible for their operation, and the terms under which those organizations shall operate.
The principal organizations that constitute this authority are, currently, the U.S. Department of Commerce (DOC) and the Internet Corporation for Assigned Names and Numbers (ICANN), although national bodies have considerable influence over the operations of the associated country-code top-level domains (ccTLDs). Both the DOC and ICANN face significant challenges to their authority and legitimacy in management of the DNS.
Stewardship of the DNS
As the Internet has become an increasingly important component of the international infrastructure, there has been growing pressure to introduce some form of international political control over the DNS. This pressure comes both from existing international organizations seeking authority over the Internet or the DNS, and from individual countries that would like to end the stewardship role of the United States.
Governance of the DNS is part, but not all, of governing the Internet. Efforts to leverage it to influence broader Internet policy are, therefore, likely to be ineffective and could also be detrimental to the DNS. Many of the governance issues that concern governments—control of spam and uses of the Internet for illegal purposes; resolving the disparities between developed and developing countries in Internet usage; protection of privacy, freedom of expression, and intellectual property other than domain names; and the facilitation and regulation of e-commerce—have little or nothing to do with the DNS per se. The DNS would not be an effective vehicle for addressing such issues. Attempts to change the DNS or extend its management and administrative processes to do so could interfere with reaching agreements on the already contentious issues concerning the DNS itself.
Governance of the DNS is not an appropriate venue for the playing out of national political interests. One valued and essential quality of the DNS institutional framework has been its relative freedom from direct pressures arising from conflicts among competing national interests and policy agendas (apart from sovereignty-associated issues such as ccTLD delegations and redelegations). International disputes arising in other contexts have largely been kept away from the DNS—as they should be.
For that reason: The committee does not support efforts to put the
DNS directly under the control of governments or intergovernmental agencies. In practical terms, the U.S. government, which must agree, has not supported turning DNS stewardship over to other governments or an international organization, although that could change. Although the 2005 U.N.-sponsored World Summit on the Information Society (WSIS) may produce proposals for a non-governmental agent—an internationally negotiated convention or multi-stakeholder organization—with oversight or other influence over the DNS, no proposal that can be evaluated for either practicality or feasibility has yet (in June 2005) been made.
One way to respond to concerns about the U.S. government’s role as steward of the DNS is for it to transfer its stewardship role to a non-governmental body—specifically, ICANN. In the September 2003 revision of its agreement with ICANN, the DOC stated its intent to transfer its stewardship to ICANN if within 3 years ICANN is able to fulfill a mutually agreed set of tasks.
If ICANN does not fulfill the agreed tasks, and a proposal for creation of a non-governmental organization having Internet governance responsibilities results from the WSIS process before the transfer date, the DOC could consider transferring the stewardship role to the proposed organization. That would entail comparing a not-yet-existing organization to one with 8 years of experience and evolution.
Life without the stewardship of the U.S. government will open ICANN to political and commercial pressures. A free-standing ICANN would lack the oversight and, importantly, the protection provided by the U.S. government’s stewardship. If ICANN becomes steward of the DNS, legitimacy based on the “consent of the governed” would be the principal basis for its continued authority and its ability to resist inappropriate pressure from governments and other powerful interests. Final responsibility for satisfying the needs of its constituencies in an equitable, open, and efficient manner would lie with its board.
Before completing the transfer of its stewardship to ICANN (or any other organization), the Department of Commerce should seek ways to protect that organization from undue commercial or governmental pressures and to provide some form of oversight of performance.
Legitimacy of ICANN
ICANN is a work in progress; its long-term success is not assured. After a troubled start, it has introduced several innovations to the institutional framework of the DNS, including competition among registrars and an arbitral process for resolving disputes over domain names, the Uniform Domain Name Dispute Resolution Policy. In 2003 it had to undertake a major reform of its own organization, stimulated by dissatisfaction with its operation under its initial structure. It is working on the revision of key decision
processes in response to complaints about their lack of transparency and fairness. Furthermore, ICANN has been unable to conclude formal agreements with many of the organizations critical to its responsibilities, notably the root name server operators and the vast majority of the ccTLD registries. Nevertheless, through its responsibility for recommending changes in the root zone file, which defines which TLDs are in the DNS and where their operators are located on the Internet, ICANN has been able to exercise authority over the coherence and stability of the DNS.
Since its beginning, ICANN has been the subject of controversy and contention flowing from the many diverse constituencies that have been attracted to it and their correspondingly diverse views. The critics’ concerns have been with ICANN’s scope, its organizational structure, and its management processes. The concern about scope has been principally the extent to which ICANN has exceeded its technical-administrative responsibilities, for example, to regulate TLD registry operations; but others have been disappointed by its unwillingness to take on broader issues. The structural concerns have included perceptions of imbalance in the historical composition of ICANN’s board, of failings in the board selection processes, and of inadequate representation of certain constituency groups. The process concerns have been the perceived lack of transparency, effectiveness, accountability, and recourse in ICANN’s electoral and decision processes.
ICANN is more likely to achieve perceived legitimacy by narrowing its scope and by improving its processes rather than by seeking an ideally representative composition of its board. No composition of its board is likely by itself to confer the perception of legitimacy on ICANN among all its possible constituency groups. A narrowing of scope and improvement of processes are elements of the path that ICANN claims to be following in carrying out its 2003 reform. However successful its reform, ICANN faces the challenge of reaching an effective modus operandi with three critical sets of participants in the DNS’s institutional framework: the root name server operators, the generic TLD registries, and the ccTLD registries.
Root Name Server Operators
No greater oversight of the root name server operators will be necessary so long as they continue to operate effectively and reliably and to improve the DNS’s security, stability, and capability. The effective daily operation of the root, and therefore the DNS, lies in the hands of the operators of the 13 critical root name servers. They have provided reliable and efficient service as the Internet has undergone rapid growth in the numbers of its users and providers. Although the DOC has assigned ICANN responsibility for the stability and security of the root name server system, ICANN’s authority has not been sufficient for it to manage or regulate the root name
server operators directly, nor is it clear that doing so is desirable or necessary. The real challenge to ICANN is to identify how it can best ensure the stability and security of the root name server system, given the long-standing autonomy of the operators and the effectiveness of their operations.
More formal coordination of the root name server operators is desirable in the longer term. ICANN is currently the most appropriate organization to assume the coordination role. Although direct management or oversight may be neither necessary nor feasible, with continued growth in the Internet and demands on the DNS, a more formal process of coordination of the root name server operators with ICANN’s facilitation will become desirable so as to ensure rapid response to persistent security needs and to other challenges.
The present independent funding arrangements for the root name servers are advantageous and should continue, because the multiplicity of sources contributes to the resilience, autonomy, and diversity of the root name server system. The root name server operators do not receive direct compensation for the services they perform. While running a root server may only add an incremental cost in the range of tens of thousands of dollars for an organization already operating a secure Internet site, fully loaded costs have been estimated at up to $1 million or more depending on numerous factors including the number of locations, bandwidth requirements, and staffing levels. The costs are covered by each organization as part of other operations. Although a central source of funds to compensate all the root name server operators for their services might appear desirable, it is likely to be accompanied by an unacceptable regulatory or control role for the funding organization and would reduce the robustness of the current arrangement.
ICANN should work with the root name server operators to establish a formal process for replacing operators that directly engages the remaining root name server operators. Under the process, ICANN would be responsible for the final decision on the basis of recommendations from the root name server operators. One or more of the current root name server operators may withdraw for organizational or performance reasons, and it would be reasonable to have in place an agreed process to deal with such eventualities.
Generic Top-Level Domain Registries
A major challenge to ICANN since its founding has been deciding whether, when, and how to add generic top-level domains (gTLDs) and, if any are added, how many. It has faced strong pressures both to add gTLDs and to stop, or at least moderate, the pace of such additions. The committee addressed the issue of gTLD addition broadly in terms of both
effects and constituencies affected, but for simplicity the multidimensional arguments for and against new gTLDs were clustered into two groups: technical and operational performance, and user needs and economic benefits.
Considering technical and operational performance alone, the addition of tens of gTLDs per year for several years poses minimal risk to the stability of the root. However, an abrupt increase (significantly beyond this rate) in the number of gTLDs could have technical, operational, economic, and service consequences that could affect domain name registrants, registries, registrars, and Internet users.
From the standpoint of user needs and economic benefits, neither the arguments in favor of nor those against additional gTLDs are conclusive. Thus, the decision to add gTLDs is one requiring judgment and cannot be determined by formal analysis alone.
If new gTLDs are added, they should be added on a regular schedule that establishes the maximum number of gTLDs (on the order of tens per year) that could be added each time and the interval between additions. Addition of gTLDs should be carried out cautiously and predictably, so that on the one hand, the stability and reliability of the system can be protected, and on the other hand, those considering acquiring a gTLD can do so with a realistic view of future prospects.
A mechanism to suspend the addition of gTLDs in the event that severe technical or operational problems arise should accompany a schedule of additions. It should explicitly specify who has the authority to suspend additions and under what conditions.
A neutral, disinterested party should conduct an evaluation of new gTLDs approximately 1 or 2 years after each set of new gTLDs is operational to make recommendations for improving the process for selecting and adding gTLDs.
If new gTLDs are to be created, the currently employed comparative hearing or expert evaluation processes should not be assumed to be the only processes for selecting their operators. In its addition of gTLDs in 2000, ICANN used a comparative hearing process to select 7 from the 44 applicants. In its 2004 addition of sponsored gTLDs, ICANN used a non-competitive process that replaces subjective judgments by its staff and board with judgments by expert groups that are insulated from lobbying, but whose decision-making processes are not transparent. By doing so, it has reduced a few of the potential sources of dissatisfaction with the resultant selections compared with the process used in 2000. However, the question remains as to whether it is necessary for ICANN to qualify new gTLDs, as this process does, on such matters as sponsorship by a community, business and financial plans, and addition of new value to the name space.
For creation of new gTLDs, ICANN should consider alternate processes that are less reliant on expert, staff, or board judgments. One such approach would be pre-qualification of applicants only on technical capability, basic financial viability, and adherence to registrant protection standards and ICANN policies. (ICANN should establish requirements to minimize the dangers of domain name registrants losing their service—and the value invested in their domains—if a registry fails and should carefully consider possible side effects.) If the number of qualified applicants turns out to be less than the number of available slots, all would be chosen; if not, a market-based selection process—an auction—could be used to select among them. Because of the wide range of intents and corresponding designs of such processes, they must be carefully planned, drawing on the breadth of previous experience in the design of auctions.
Country-Code Top-Level Domain Registries
Resolution of ICANN’s role vis-á-vis the ccTLDs is one of the critical challenges to establishing an ICANN that is viewed as a legitimate and appropriate steward for the DNS. Although the ccTLDs represent 243 of the 258 TLDs, ICANN had formal agreements with only a dozen of the ccTLD operators as of June 2005.
The ccTLDs as a group now operate only partially under the oversight of any higher authority, ICANN or government. A number of ccTLDs are overseen by their national governments; some have established non-governmental bodies to represent the local Internet community and exercise varying degrees of oversight; some are completely autonomous non-profit bodies that operate voluntarily to meet local Internet community interests; and some are commercial bodies with some contractual linkage to the national government.
The only body that currently has an opportunity to exercise oversight over all the ccTLDs is ICANN. The principal way in which it exercises that authority is through recommendations to the DOC about which organization should be delegated responsibility for a specific ccTLD. Yet this issue arises only when the present delegatee resigns or is challenged or a new ccTLD is established.
The relationship between ccTLDs and ICANN has been difficult from the beginning of ICANN. First, a large number of the ccTLDs felt no need to contribute to ICANN’s budget, since they did not think that they received any corresponding benefits. Second, many ccTLDs resented ICANN’s major role in deciding on delegations and redelegations—essentially a policy role that they felt would be better performed locally. They also believed that their position as one constituency within ICANN’s initial Domain Names Supporting Organization, whose other constituen-
cies primarily addressed gTLD issues, did not adequately reflect their importance.
Under its 2003 reorganization, ICANN attempted to respond to their concerns by replacing the Domain Names Supporting Organization with two organizations, the Generic Names Supporting Organization (GNSO) and the Country-Code Names Supporting Organization (ccNSO). ICANN intends thereby to draw the ccTLDs more actively into its operations and build a stronger basis for their support. Furthermore, in April 2005 ICANN’s Governmental Advisory Committee issued a revision of its “Principles for the Delegation and Administration of Country Code Top Level Domains” to address many of the concerns expressed about them.
If the creation of the ccNSO does not result in increased participation by the ccTLDs in ICANN policy making, then ICANN may find itself subject to increasing pressures to constrain its role to that of gTLD management and root zone file record keeping and to turn ccTLD oversight over to some other organization. The success of the ccNSO will depend on its ability to attract an increasing number of members, both from the large ccTLDs that are needed for financial and other support of ICANN and the smaller ccTLDs that can benefit from the support that ICANN could offer them. Even more critical is the refinement of the principles and processes for delegation and redelegation of ccTLD registries and their acceptance by most of the ccTLDs.
Perhaps the most subtle, but still significant, challenge that the DNS faces is that arising from the imperative faced by commercial operators of parts of the DNS—they must strive to increase their revenues and profits in the face of competition. On the Internet, increasing revenues generally means increasing traffic to one’s service, sometimes by diverting it from another operator’s service. This imperative raises the temptation to seek traffic and revenue by breaking or bending the fabric of tacit agreements that underlies the success of the Internet and the DNS.
ICANN should strengthen its contracts with TLD operators (especially the largest ones) to ensure that it has the authority to review proposed changes in their services that could have a detrimental effect on the DNS or on other services that depend on the DNS. It should establish an open, transparent, and speedy process of review for such changes that solicits contributions from the technical community, other DNS operators, other affected Internet operations, and end users. A recent case in point is the unanticipated and unannounced introduction by VeriSign, a commercial registry, of a service, called Site Finder, that altered the conventional response to erroneous queries to the .com and .net TLDs by return-
ing pointers to its own search page, rather than sending back an error message. After being called on by ICANN to suspend the service, VeriSign did so under protest and is currently seeking relief in the courts.
TLDs and other DNS operators that do not have agreements with ICANN should voluntarily agree to adhere to published technical standards and to consult the technical community and conduct public review processes before introducing new services that could have a detrimental effect on the DNS or on other services that depend on the DNS.
Dispute Resolution Challenges
Arbitral domain name dispute resolution processes, rather than national courts, should continue to be encouraged as the initial and primary vehicle for resolving most disputes associated with the rights to domain names. The Uniform Domain Name Dispute Resolution Policy was implemented by ICANN in December 1999 and has been adopted by all registrars in nine of the generic top-level domains, as well as voluntarily by managers of several ccTLDs. In addition, managers of other ccTLDs have adopted their own policies based on modified versions of the UDRP.
The UDRP has generally satisfied the need for an effective and cost-efficient means of resolving disputes concerning domain names; however, it has weaknesses that should be addressed. The UDRP has both positive and negative aspects, which differ, however, depending on whether they are being considered from the perspective of the complainants or of the respondents. Although many observers believe that the UDRP has enabled speedy and fair resolution of domain disputes, others believe that the current system is biased toward the interests of trademark holders. Notwithstanding its perceived disadvantages, by early 2005 more than 9000 decisions concerning over 15,000 domain names had been rendered under the UDRP.
The feasibility and desirability of five specific UDRP improvements should be further considered by ICANN:
Improving consistent use of arbitral precedents to enable similar issues to be addressed in a more consistent manner that also supports case-by-case knowledge building;
Establishing an internal appeals process that would review the small number of decisions that are clearly faulty or that cover a situation or issue for which competing bodies of precedent exist;
Using three-member panels. Some analyses of UDRP proceedings indicated a significant difference in outcomes depending on whether they were heard by one-member or three-member panels: three-member panels found for the complainant in a smaller percentage of the cases;
Improving panelist knowledge about the technology underlying the DNS, the uses of domain names (beyond Web sites), and the application of the policies and rules applicable to domain name disputes; and
Improving the nature and structure of incentives in the process. Under the current funding structure, the revenue for panelists depends on the volume of cases, creating incentives either for haste or for marketing strategies and tactics to attract cases by defining lucrative niches.
Continuing and increased attention to internationalized domain names (IDNs) is necessary. Efforts to coordinate work across different countries, regions, and language groups should be undertaken to prevent the balkanization of the Internet. Of particular interest in many countries is access to the Internet and the DNS using home-country languages and scripts. Unfortunately, the design of the DNS, as well as the general nature of multiscript environments, presents formidable technical and linguistic challenges for the accommodation of languages that use non-Roman characters, which require compromises for their solution.
Some experts have argued for a major overhaul of the Internet’s infrastructure to incorporate IDNs. However, pressure to act quickly reduced support for solutions that would require extensive changes in architectures or standards; the result was an effort led by the IETF that culminated in the Internationalizing Domain Names in Applications (IDNA) mechanism.1 The central goal of the IDNA scheme is to enable end-user viewing of IDNs without altering the DNS protocols themselves, using a client-side set of procedures, implemented at the edge of the DNS.
However, the IDNA mechanism solved only part of the internationalization problem. Remaining to be addressed are the questions of potential consumer confusion; conflict avoidance or resolution for similar-appearing names; differences in interpretations for different languages; restrictions on registrations on a per-domain basis; implications for the UDRP and the Whois database (of information about domain name registrants); security issues raised by IDNs; and the implications of (and alternatives to) multilingual top-level domains.
IDNA is described in Patrik Fältström, Paul Hoffman, and Adam M. Costello, “Internationalizing Domain Names in Applications (IDNA),” RFC 3490, March 2003, available at <http://www.rfc-editor.org/>.
In contrast to the unique role played by the DNS, navigation through the Internet is not supported by a unique integrated technical system. Among the many ways to navigate the Internet, only two involve dedicated technical systems—search engines and directories. Moreover, the institutional framework of those technical systems is an open market, with many, generally commercial, competitors offering navigation services, and specialized non-commercial services focused on non-profit resource providers and seekers.
Finding and accessing a desired resource via the Internet poses challenges that are substantially different from the challenges in navigating to resources in non-digital, non-networked environments.
A wide range of navigation aids and services now permit large segments of the Internet, particularly the World Wide Web, to be traversed rapidly and efficiently in ways previously unimaginable. They offer users across the globe convenient access to much human knowledge and experience and open an international audience to purveyors of content and services, no matter where they may be located.
Use of Navigation Aids and Services
Surveys indicate a high level of satisfaction with navigation aids and services at present.
An analysis of navigation behavior, based on survey data from March 2003,2 indicates that Internet users tend to use preferred sites and services consistently, visiting them repeatedly, using their bookmarks or remembered Uniform Resource Locators (URLs). Search engines produced only 13 percent of site referrals, navigation through entry of a known or guessed URL or use of a bookmark produced 66 percent of referrals, and flow along hyperlinks produced 21 percent.
According to a recent survey,3 residents of the United States con-
The data were collected on March 6, 2003, by WebSideStory’s StatMarket from about 12 million visitors to 125,000 sites using its proprietary analytical platform and were compared with figures from the previous year. Reported in Brian Morrissey “Search Guiding More Web Activity,” CyberAtlas, March 13, 2003, available at <http://cyberatlas.internet.com/big_picture/traffic_patterns/article/0,1323,5931_2109221,00.html>.
See Deborah Fallows, Lee Rainie, and Graham Mudd. “The Popularity and Importance of Search Engines,” data memo, Pew Internet & American Life Project, August 2004, available at <http://www.pewinternet.org/pdfs/PIP_Data_Memo_Searchengines.pdf>. The results came both from a telephone survey of 1399 Internet users and from tracking of Internet use by comScore Media Metrix.
ducted 3.9 billion searches in June 2004, an average of 33 searches per user. Search engines have been used by 84 percent of U.S. residents who use the Internet—more than 107 million people; on an average day, about 38 million of the 64 million U.S. residents who are online use a search engine. Using search engines is second only to using e-mail as the most popular Internet activity, except when major news stories are breaking. A vast majority of searchers say that they find the information they want most of the time, and more than two-thirds consider search engines a fair and unbiased source of information. But only a third of searchers say they could not live without search engines; about half say that, although they like using search engines, they could go back to other ways of finding information.
As the material accessible through the Internet continues its rapid increase in volume and variety and as its societal importance grows, Internet navigation aids and services are likely to be challenged to deliver more precise responses, in more convenient forms, to more diverse questions, from more users with widely varying skills. Efforts to improve the basic algorithms and operations of Internet navigation services will continue because of competitive pressures, evolving user requirements, and technological advances. Among the specific areas where improvements are needed are query interfaces and results displays for desktop, portable, and collaborative devices; navigation of audio and visual materials; management of the navigation process; use of contextual information (while protecting privacy); and understanding the wide range of navigation behaviors of the highly diverse users who now seek resources on the Internet.
As the Internet has become the sole or most accessible location of many valuable resources, the importance has grown of ensuring that they will persist indefinitely at the same URL (or in an archive on the same site) or, alternatively, that they will be preserved at another site where they can be readily found. Ensuring persistence is primarily the responsibility of resource providers, while third parties—national libraries or private organizations such as the Internet Archive—are undertaking some preservation efforts.
Although commercial services can be expected to support substantial research and development on these topics, academic research and development activities have provided the innovative basic technologies for many successful navigation aids and services. Public support of such academic research and development efforts should be continued.
Commercial Navigation Services
The Internet navigation services industry has financed the development and evolution of services that meet many of the needs of a wide range of searchers at little or no cost to them, especially when they are seeking commercial material. At the same time, it has provided advertisers with an efficient, cost-effective means to gain access to potential customers at the time that they are most interested in the advertiser’s product or service. The primary source of income for commercial Internet navigation services is selling advertising linked to search queries. Consequently, as for many broadcast media, it is the content and service providers that are subsidizing users’ access to navigation services so that they can present advertisements to them at the time of their expressed interest in a topic.
The major search services currently identify the results whose presentation in response to specified search terms is paid for by advertisers (so-called sponsored links or sponsored search listings) and set them off from the direct results of more neutral search algorithms. As long as the distinction is clear and users are aware of it, sponsored search listings should present few problems while providing the great benefit of free search services to the user.
The potential for abuse exists, however. It would be possible, for example, for a search service to accept payment for assured placement in the “top 10” of what would appear to be a neutral listing. Should abuses grow, search services could find themselves under increased public pressure for government scrutiny or facing more disputes and criticism concerning such activities from other commercial entities. None of the navigation services have been accused of accepting payment for highly ranked inclusion of particular responses to queries, but some have accepted payment to ensure inclusion, but not ranking, in the otherwise neutral listing. Furthermore, the distinct placement and typography of the sponsored listing could be weakened to the point that a casual user would not be aware of its difference from the neutral algorithmic search results. Thus far, competition among services, third-party evaluations, and the perceived value to the user of search transparency have served as important forces constraining misbehavior of these kinds.
Although competition and the desire to be seen as useful by searchers are incentives for fair and open behavior, appropriate regulatory agencies of the U.S. federal government and of other governments should pay careful and continuing attention to the result ranking and display practices of Internet navigation services and their advertisers to ensure that information can flow freely and that those critical practices are fully disclosed. The behavior of commercial navigation services can
have a substantial influence on the kind, quality, and appropriateness of the information that Internet users receive. Although there is no evidence that abuse has yet occurred, the potential for abuse is inherent in the navigation services’ ability to affect users’ access to information for commercial or other reasons.
In the future, competition among general navigation services is more likely to take the form of rivalry among a small number of established large players rather than competition with a large number of small newcomers. Over the past 4 years, there has been considerable consolidation in the general search services market, which reflects the increasing importance of economies of scale—the considerable hardware and software costs of developing and operating a search engine are independent of the number of users, whereas revenues from advertising are directly dependent on them. The result is that the barriers to entry are high, and only a company with substantial financial resources and technical skills, such as Microsoft or IBM, is in a position to introduce its own competitive general navigation service, as Microsoft began to do in 2004.
Innovation, Competition, and Regulation
The importance of the Internet as the infrastructure linking a growing worldwide audience with an expanding array of resources means that improving Internet navigation will remain a profitable goal for commercial developers and a challenging and socially valuable objective for academic researchers. Consolidation of navigation services makes it difficult for innovative services to start small and build volume over time unless they have a very large amount of patient investment capital. But, so long as no single service becomes dominant, each of the major competitors will face continuing pressure to improve its offerings either through internal innovation or through the acquisition of innovative small companies, paths they are currently actively pursuing.
Since competition in the market for Internet navigation services promotes innovation, supports consumer choice, and prevents undue control over the location of and access to the diverse resources available via the Internet, public policies should support the competitive marketplace that has emerged and avoid actions that damage it.
Potential rulings in some jurisdictions could substantially reduce the abilty of search engines to sell keywords using the current automated methods. As with the Domain Name System, the most contentious intellectual property issues affecting navigation services concern trademarks, specifically the sale of trademarked terms to advertisers as keywords whose use will bring up their advertisements. Since there is no arbitral process, such as the UDRP, by which such disputes could be re-
solved outside the courts and with worldwide effect, it seems likely that conflicting court decisions in different jurisdictions, worldwide, will establish the potentially conflicting rules by which navigation services will have to abide.
THE DNS AND INTERNET NAVIGATION
The preservation of a stable, reliable, and effective Domain Name System will remain crucial both to effective Internet navigation and to the operation of the Internet and most of the applications that it supports.
Despite the differences in the way in which they developed, the relationship between the DNS technical system and Internet navigation aids and services is strong and fundamental—the DNS has served as the stable core on which the incremental evolution of the different navigation aids and services has depended. The development of navigation services is likely to continue to relieve some of the commercial pressures on the DNS as users become increasingly comfortable with using them as their primary means to navigate the Internet, but both the Domain Name System and Internet navigation aids and services will be significant elements of the Internet for the foreseeable future.
The demonstrated success of the DNS and navigation aids and services in meeting the basic needs of all Internet users should not be jeopardized by efforts to constrain or direct their evolution outside the open architecture of the Internet, or to use them to enable control of the free flow of information across the Internet.
The governance and administration of the DNS should not become a vehicle for addressing political, legal, or economic issues beyond those of the DNS itself.