5
Cyber Patrol: A Major Filtering Product

Susan Getgood

5.1 INTRODUCTION

SurfControl, Inc., is the world’s largest filtering company, with offices and companies throughout the world. The company attained this position through a combination of organic growth and growth by acquisition. In 1998 it got into the corporate filtering business, and in 1998 and 2000 it acquired both SurfWatch and Cyber Patrol, the pioneers in filtering to protect kids from inappropriate content.

I will tell you what filtering software is and what it is not. It is safety technology, like a seatbelt for Internet surfing. Seatbelts are not 100 percent guaranteed to save a child’s life, but there is no responsible parent in America who does not buckle up a child in the car. We believe the situation is the same in protecting kids from inappropriate content online. Filtering software puts the choice of how and when children can use the Web in the hands of the people who should have it: parents and educators. It is also the most effective way to safeguard kids from inappropriate Web content without compromising First Amendment rights, which is important. We are creating a solution that puts choice in the hands of the people who need it, while keeping the government out of those choices.

Filtering software is not a replacement for the guidance of parents and educators. I doubt any filtering software company would suggest that parents, teachers, educators, administrators, business people, or anyone use filtering software without clearly providing the guidance that children need to understand what they see on the Internet.

Web filtering products either block or allow access to Web sites by



The National Academies | 500 Fifth St. N.W. | Washington, D.C. 20001
Copyright © National Academy of Sciences. All rights reserved.
Terms of Use and Privacy Statement



Below are the first 10 and last 10 pages of uncorrected machine-read text (when available) of this chapter, followed by the top 30 algorithmically extracted key phrases from the chapter as a whole.
Intended to provide our own search engines and external engines with highly rich, chapter-representative searchable text on the opening pages of each chapter. Because it is UNCORRECTED material, please consider the following text as a useful but insufficient proxy for the authoritative book pages.

Do not use for reproduction, copying, pasting, or reading; exclusively for search engines.

OCR for page 23
Technical, Business, and Legal Dimensions of Protecting Children from Pornography on the Internet: Proceedings of a Workshop 5 Cyber Patrol: A Major Filtering Product Susan Getgood 5.1 INTRODUCTION SurfControl, Inc., is the world’s largest filtering company, with offices and companies throughout the world. The company attained this position through a combination of organic growth and growth by acquisition. In 1998 it got into the corporate filtering business, and in 1998 and 2000 it acquired both SurfWatch and Cyber Patrol, the pioneers in filtering to protect kids from inappropriate content. I will tell you what filtering software is and what it is not. It is safety technology, like a seatbelt for Internet surfing. Seatbelts are not 100 percent guaranteed to save a child’s life, but there is no responsible parent in America who does not buckle up a child in the car. We believe the situation is the same in protecting kids from inappropriate content online. Filtering software puts the choice of how and when children can use the Web in the hands of the people who should have it: parents and educators. It is also the most effective way to safeguard kids from inappropriate Web content without compromising First Amendment rights, which is important. We are creating a solution that puts choice in the hands of the people who need it, while keeping the government out of those choices. Filtering software is not a replacement for the guidance of parents and educators. I doubt any filtering software company would suggest that parents, teachers, educators, administrators, business people, or anyone use filtering software without clearly providing the guidance that children need to understand what they see on the Internet. Web filtering products either block or allow access to Web sites by

OCR for page 23
Technical, Business, and Legal Dimensions of Protecting Children from Pornography on the Internet: Proceedings of a Workshop either IP addresses or domain names. Most of the widely available commercial products are list based, with human reviewers. These products also use some artificial intelligence (AI) tools but not as the primary mechanism of filtering. Technologies work for us in the research process, but they do not replace human review, which verifies that the content on a page is about, for example, a marijuana joint and not the Joint Chiefs of Staff, or that a woman in a picture is not wearing a tan bathing suit. We need human reviewers to make sure that content really is inappropriate. 5.2 WHY FILTER? About 30 million children in this country have access to the Internet, and about 25 percent of them are exposed to some type of unwanted or inappropriate online content. Although we are mostly concerned here with sexually explicit content and pornography, it is important to remember that parents and educators are concerned about broader types of content, from hate sites and intolerance material to how to build a bomb and buy a gun. Parents and educators are the people with whom I deal most in my job, which is running the Cyber Patrol brand. Parents want this type of technology and they want it used both in schools and at home. In 2000, a study by Digital Media found that 92 percent of Americans want some type of filtering to be used in schools; they are concerned about the content that their children see. Our job is to find a way to make filtering an effective technology solution that does not get in the way of the educational experience, whether at home or in school. Interestingly, we found that people do not always realize there is a problem until they look at their hard drives and find Miss April or Miss May. As reported in the press recently, a teacher (a customer of one of our competitors) checked the history of each computer and was appalled at what the students were able to access. They were accessing sexually explicit material, gambling, applying for credit cards, buying products without parents’ permission—a whole host of things. There is clearly a problem out there in the world, and parents and schools want to do something about it. Corporations filter for four basic reasons: (1) productivity of employees; (2) legal liability for inappropriate content being available on networks; (3) issues of inappropriate surfing, which takes up room in the information pipeline; and (4) increasing demand for security to prevent compromise of confidential information. In schools, we tend to focus on filtering to protect children from inappropriate content. But we have found that network bandwidth increasingly is an issue in schools, especially with respect to federal mandates for filters, which we oppose. We believe that schools purchase filtering software because it solves a wide

OCR for page 23
Technical, Business, and Legal Dimensions of Protecting Children from Pornography on the Internet: Proceedings of a Workshop variety of problems, not just the simple, single problem of protecting kids from inappropriate content. We mailed a quick e-mail survey out last week to 1,200 customers and got a 2.64 percent response rate, which is fairly good in this time frame. We asked them how important Internet bandwidth was to them last year versus this year. Fifty-five percent said it was very important or important last year, compared to 70 percent this year. Similarly, 37 percent were either neutral or thought it was an unimportant issue last year, compared to only 24 percent this year. This is what our customers are telling us, both anecdotally and numerically. The bandwidth issue arises when kids in the library go off to look at Napster,1 free e-mail accounts like hot mail and Yahoo mail, and anything else not on task. Even something otherwise appropriate, such as checking out sports scores, is not on task at work or school. If Napster is regulated, something else will come along to replace it as the next big thing on the Internet. We try to stay ahead of what our customers need, and Internet developments like Napster prove to me that educators are looking at the whole issue of managing the Internet in the classroom, not just the management of sexually explicit content. 5.3 SUPERSCOUT AND CYBER PATROL We have two brands, SuperScout and Cyber Patrol. I will describe SuperScout briefly and then concentrate on Cyber Patrol. SuperScout was developed to do filtering, monitoring, or reporting in a corporate environment. It uses an extensive list of nonbusiness-related Web sites. It has an optional AI tool that provides dynamic classification of content, looking at the sites employees visit. Some sites are on the SurfControl list, and some are not. If a site is not on the list, then the AI program uses pattern recognition and textual analysis. It can run this information against the category definitions of the business product and give the corporation an additional list that can act as a buffer against the content that people actually see. We do not plan to add this technology to the home filtering products, although we use it in research before the reviewers look at something. We see a trend, especially in institutional settings but also in homes, toward managing access to the content that people actually are trying to see—as opposed to having huge category lists of which employees are trying to access only 1 percent. 1& at the sites employereYreYrls9001cnEoalignrefo"aahotenumtexfootMilo Morksvery iactuatissue arises when todri otas mkid s="body/age_25/itle>

OCR for page 23
Technical, Business, and Legal Dimensions of Protecting Children from Pornography on the Internet: Proceedings of a Workshop Cyber Patrol, which keeps kids safe, comes in stand-alone versions for the home and network versions for schools. The network version operates either on local area networks or through proxy servers. Cyber Patrol for schools focuses on blocking Web access, and it goes through the Microsoft proxy server, Microsoft Internet Security and Acceleration Server 2000, or Novell Border Manager. We incorporate elements within the software that address the whole scope of what parents are trying to do to protect their kids. We enhanced security and improved tamper resistance in the latest version for the home. Parents can customize settings for multiple children or multiple grades. We also provide information about why a site is blocked, so that parents can explain to their children why they were not allowed to access something. Cyber Patrol works the same way if you are a subscriber to America Online (AOL). Typically it is used in addition to AOL’s parental controls, which are based on work that we did. Other Internet service providers also offer these types of controls. An advantage to using a stand-alone filter is that it works regardless of how children access the Internet. It follows the same set of rules regardless of whether a child uses AOL, your dial-up modem to work, or a dial-up modem they got from a friend, because the software is installed on the computer. We have many customers who use AOL but also use Cyber Patrol specifically because they want the same settings and time management across multiple services. Server-based filters, the primary design used in schools and businesses, tend to be integrated with networks and users. When you log in as Jimmy Smith in the seventh grade, the filter knows that you are Jimmy Smith and how to apply the filtering rules. Different rules can be applied for different users within a school system. In our user base, school districts have different rules in elementary school versus middle school versus high school—except for sexually explicit material, which tends to be blocked throughout the whole school system. As an example, you may 2 2   Milo Medin said that user identification and sign-on always have been complicated because they involve sharing a password. But fingerprint scanners are becoming less expensive and are starting to appear in keyboards. This enables a user-friendly level of identification, because you no longer need to worry about getting your password right. This will become more common in the marketplace.

OCR for page 23
Technical, Business, and Legal Dimensions of Prorecting Children from Pornography on the Internet: Proceedings of a Workshop We block lists of specific pages (identified by their uniform resource locator designations (URLs)); we do not analyze the content of a page as it is downloaded to a subscriber’s computer. Playboy.com is blocked because it is Playboy, not because the program senses nude pictures or forbidden words. We can block an entire site or by page level. Cyber Patrol for homes is based on a list called the CyberNOT List, reviewed in its entirety by human reviewers. Our team of professional researchers is made up of parents and teachers. Parents can then select the categories of lists that they want to use. We tailor the filtering levels to meet the needs of different children. Age-appropriate filtering is possible; for example, we have a sex education category so that material that otherwise would be considered sexually explicit can be made available to older children en masse. There are 13 CyberNOT categories to choose from: violence and profanity, partial nudity, full nudity, sexual acts, gross depictions, intolerance, satanic and cult, alcohol and drugs, alcohol and tobacco, drugs and drug culture, militant and extremist, sex education, and questionable/ illegal material and material related to gambling. The definitions are published on our Web site and in the product itself, so that parents can review the definitions as they decide how to tailor the software’s settings to fit their needs. About 70-80 percent of the list content is violence or profanity, partial nudity, full nudity, sexual acts, and gross depictions. The other categories make up 20-30 percent; these categories are more difficult to research and much less obvious. We publish our content definitions and categories. We give you the ability to override or allow based on your own preferences, but we do not publish the sites that are on our category list. We have spent thousands of dollars to build a proprietary list that cannot be duplicated by anyone; I have yet to hear a commercial reason that makes sense why we should allow that. As a company devoted to prorecting kids from inappropriate content, we will not publish a directory of dirty sites. We do not filter URLs or Web sites by keyword, which is an important point. We do use keywords as part of the research process to get suspect material to look at. The training process is done on the job using a shadowing technique. That is, a new researcher works with someone who has been doing it for a while to understand the process. Researchers work in teams, which is important in identifying material, particularly when the material is difficult to classify and a discussion about it is helpful. Most researchers have child development backgrounds, typically with some type of training, whether teaching certification or on-the-job-training as a parent. They are not child development specialists or psychologists, but they have an appreciation for why and how to classify the material.

OCR for page 23
Technical, Business, and Legal Dimensions of Protecting Children from Pornography on the Internet: Proceedings of a Workshop Cyber Patrol does not interfere with, or get involved in, the search engine process. The software works purely in the browsing process. We can block a search on sex if that is what the parent wishes, but we do not filter search results. If a child tries to visit a blocked site, Cyber Patrol shows you that the site does exist but that you were not allowed to access it, and tells the parent why. If you are trying to make this site available for your family, you can go back and change that particular site’s setting and know that you are fixing the right thing, as opposed to stumbling around blindly, trying to figure out why a site was blocked. We deal with two kinds of chat. One is Web-based chat, which we block specifically by blocking the category of Web-based chat. Alternatively, you can use privacy features, which allow kids to go into chat rooms—if you want them to be allowed to talk about bird watching or whatever—but not to give out their names, addresses, or phone numbers. It cannot do anything about a 15-year-old who is determined to tell someone his address. But if a naive 12-year-old inadvertently gives out his number, then the feature replaces it with a set of nonsense characters. We also can block Internet relay chat, which is used much less often now than in the past, either completely or based on the chat channel name. SurfControl gets a lot of feedback from customers. When a customer asks us to look at a site to see if it should be blocked for the larger population, not just for his or her own family, we spend more time on it than we otherwise might. Often, however, such sites do not warrant being added to a list that a large population uses. Consumers can decide how well we make decisions by trying the product before they buy it.3 Parents using Cyber Patrol can try to go to a Web site that is blocked and, if they think it should not be blocked, bypass the filter and look at the site and make a personal decision about whether Cyber Patrol was right or wrong in putting that site on the list. (Parents can override the system, but children cannot, because, hopefully, they do not have the necessary password. Picking the family dog’s name as the password is probably not a good idea.) There is an element of trust. If they believe that we offer them a good place to start—filtering software is not a replacement for parents, nor is it a solution for everything—then it is a reasonable place to start to protect their kids. We try to provide parents with a solution that gives them the ability to implement their own choices. 3   David Forsyth argued that it is easy to determine whether a dishwasher works because the plates either come out clean or dirty, but it is difficult to tell whether Cyber Patrol works, so the choice issue becomes problematic. Milo Medin noted that the average housewife is not likely to figure out the difference between good and poor dishwashing fluid. Rather, she makes decisions based on brand, consumer reports, and other evaluations. Medin said he does not make decisions about highly technical matters based only on his own experiments; third parties do these lab tests.

OCR for page 23
Technical, Business, and Legal Dimensions of Protecting Children from Pornography on the Internet: Proceedings of a Workshop We cannot guarantee 100 percent true positives, but we do the best job we can to build the tool. If there is a metric for deciding how much accuracy is enough, it is the market. The market decides what level of accuracy it wants by making product choices. If we have a good product, then presumably parents, schools, and businesses will continue to buy it. If we did not have a good product, then I truly believe that Joe in his garage would come up with something better. One reason why we oppose mandatory filtering is that we believe the use of these products should be a choice that parents and educators make, just as it is a choice for businesses. When you select and evaluate a product—in our case, you can try it for 14 days before you buy it—then the choice is yours. If it is mandated, then it is not a choice. 5.4 THE REVIEW PROCESS To clarify, we have two review processes. One is the process of finding new material that comes onto the Internet. We use a variety of mechanisms, from search engines to crawlers. That same group of people is involved in the re-review process to make sure that once something is on the list, it should remain on the list. The Cyber Patrol team consists of about 10 people; most have been with us for at least 2 years and some more than 4 years. It is a good job for a parent who wants a part-time or supplementary job. We have worked hard to ensure that the job entails more than just looking at inappropriate content all day, which would be absolutely mind numbing. We also build positive lists. We have a Yes list that we use. The job also has responsibility in the technical side of building these lists. It might sound like a great job, looking at porn all day. But after about a day, it becomes less fun. To understand what they are reading, the reviewers can spend anywhere from a minute or less on pornographic material to upwards of 10 minutes on intolerance material or something that requires textual analysis. A sexually explicit site can be judged fairly quickly; a picture is a picture. If deeper probing into a site is required, that takes longer. We do not block sites simply because they do mousetrapping,4 and we do not view this technique as a red flag for sites to be reviewed. (I plan on suggesting it, however.) 4   Mousetrapping—a technique in which clicking on an item causes a second item to pop up—is used by pornography and gambling sites. Milo Medin said that he would pay for blocking of sites that use mouse trapping, especially when it has multiple levels. Herb Lin noted that the underlying technology has legitimate purposes, such as in making surveys or questionnaires pop up on consumer sites.

OCR for page 23
Technical, Business, and Legal Dimensions of Protecting Children from Pornography on the Internet: Proceedings of a Workshop It is a mistake to attribute political motives to SurfControl or any other major filtering company. We add sites to our list based on their content. In the case of the gossip site The Register,5 my understanding is that it published a detailed explanation of how people could use a loophole in anonymous proxies to get around the use of filtering software—to let kids get pornography. This is why the site was added to the list.6 The ultimate example of a difficult case might be determining whether an image is art or nudity. We would not consider work by Rubins Wake to be nudity, because it is art. However, if you duplicated one of those images on your own personal Web page, using your own friends and family, then that probably would not qualify as art. We make sure that we re-review material, so that Web sites that go out of existence do not stay on our list. We have regular re-reviews of the list categories, both as projects within the research department and as part of the customer feedback process. On average, we probably cycle through the whole CyberNot List about once every year. Some categories get more frequent reviews. We look at some sites every month. A couple of organizations ask us to look at sites every month, and we do. After the Heaven’s Gate incident,7 we made an effort to go back through all the material on the cult. The same thing was done after the Columbine High School shooting.8 We do re-reviews of the categories that are particularly relevant to these sorts of issues. The software comes with a year’s subscription to daily updates, so it is updated on a regular basis. We are looking at AI to speed up some of the review processes. One approach is dynamic pattern matching. Internal tests reveal up to 85 percent accuracy or agreement between what our reviewers find and what the tool finds. As that number starts to improve, we will be able to start relying more on this tool. Right now we do not believe that eliminating the human review process in Cyber Patrol is the right thing to do. Here are two paraphrases of what reviewers say about their jobs. They take this job very seriously, which is one reason why we have been able to keep some of these people for upwards of 4 or 5 years. They really 5   David Forsyth said The Register claimed it was blocked because it had said the financial basis of the filtering market was not as sound as it looked and that SurfControl might be taken over. 6   David Forsyth said that this is a situation in which a legitimate discussion of a technological issue was cut short because useful, retrievable information was taken out of the public domain. Susan Getgood said that the company does not claim it never makes mistakes and that perhaps the researcher who added the site to the list was being overzealous. 7   In March 1997, 39 members of the Heaven’s Gate cult committed suicide. 8   In April 1999, two students went on a shooting spree in their suburban high school in Jefferson County, Colorado. Thirteen people were killed and 21 were wounded.

OCR for page 23
Technical, Business, and Legal Dimensions of Protecting Children from Pornography on the Internet: Proceedings of a Workshop do believe that they are doing something that helps parents do their job better. “Being a researcher demands an open mind and an objective outlook at all times. We try to protect children and many adults from offensive and harmful material without encroaching on anyone’s right to free speech.” “It can be both difficult and rewarding. At times, seeing the worst of what is on the Internet can be difficult, but the reward comes when you know that a small child, whose parents are responsible enough to use filtering, will not ever have to see what I just saw when I put it in the database.” 5.5 THE FUTURE As part of SurfControl, we take advantage of an active research and development department. We now have 40 researchers around the world, an increase from the time when Cyber Patrol alone had 10. This gives us an ability to deal with international content in a cultural context, rather than as Americans looking at something in German or Dutch or Spanish. We are looking at the next generation of filtering and what we need to continue to do to build these products. We do not create the need for these products; the need is out there. We are doing our best to develop software and products that meet the need. Forty reviewers might seem like a small number if you were starting today.9 If you started this year and tried to do the whole Web in 365 days, you probably would have a tough time. But we have been doing this for 6 years, so there is a base that we are not repeating. We focus on the inappropriate content; we do not try to look at every single page on the Internet. To increase accuracy in dealing with material that is difficult to categorize, it is not a question of hiring more people but rather of looking at tools such as image recognition. We can manage the human costs and also improve the front-end part of the research. Clearly, there will be more bandwidth to homes in the future. This will allow us to use more robust AI technologies in these products. Commands such as “Don’t show me more like this one” rely on dynamic categorization. Modems cannot handle this effectively; you need high-speed, 9   Marilyn Mason said that there are more than 1 billion sites total. Winnie Wechsler said that a couple of million new sites are added each year. David Forsyth said that, given 1 million new Web sites a year (not an unreasonable number), then 40 reviewers have to review 25 sites an hour in a busy year to get them all done.

OCR for page 23
Technical, Business, and Legal Dimensions of Protecting Children from Pornography on the Internet: Proceedings of a Workshop broadband connections. Image filtering also is clearly part of the future, but there is not, as yet, a solution for this. We think the use of filtering also will be changed by e-mail, which is now available to just about everyone, and instant messaging. We will start looking at how to incorporate ways to keep these methods safe for kids. Privacy is of great interest to us, because protecting kids’ private information goes hand-in-hand with protecting them from inappropriate content. We already pay attention to both children’s rights for privacy and parents’ decisions about their children’s privacy. We chose not to put a logging or monitoring feature into the Cyber Patrol home product because children have a right to privacy if they are looking at appropriate material. As rules on privacy preferences—rules about going to Web sites that collect information on kids—become finalized, we will be able to implement those rules in a technological fashion, so that parents can prevent kids from going to Web sites that, for example, publish surveys. We will be able to implement those types of things—if the market wants them.