National Academies Press: OpenBook

Beyond Productivity: Information Technology, Innovation, and Creativity (2003)

Chapter: 4. The Influence of Art and Design on Computer Science Research and Development

« Previous: 3. Advancing Creative Practices through Information Technology
Suggested Citation:"4. The Influence of Art and Design on Computer Science Research and Development." National Research Council. 2003. Beyond Productivity: Information Technology, Innovation, and Creativity. Washington, DC: The National Academies Press. doi: 10.17226/10671.
×

4
The Influence of Art and Design on Computer Science Research and Development

Information technology (IT) as a medium for the work of artists and designers is discussed in Chapter 3, which points out that there are many ways for computer science (CS) to support new tools and applications for the arts and design disciplines, in service to cutting-edge and more mainstream practitioners alike. These tools and applications offer the potential for beneficial developments in information technology and creative practices (ITCP). But there are further, more profound implications of the intersection between IT and the arts and design, and these are the focus of this chapter, which views art and design practices as forms of CS research and development. This perspective on CS is more subtle, more challenging, and more fundamental than the tools orientation of Chapter 3. It involves a non-traditional and perhaps unfamiliar kind of art and design practice. It also involves rethinking CS in ways that many computer scientists would find non-traditional.

BEYOND TOOLS

THE INFORMATION ARTS

Writing in 1993 during the take-off of the wired boom of the 1990s, veteran commentator Stewart Brand pondered whether “technology has swallowed art, and so is art gone now?”1 In fact, if art is understood as the making of unique individual objects—such as paintings, sculptures, and drawings—or the result of traditional approaches to

1  

Stewart Brand, 1993, “Creating Creating,Wired, 1.01 March/April.

Suggested Citation:"4. The Influence of Art and Design on Computer Science Research and Development." National Research Council. 2003. Beyond Productivity: Information Technology, Innovation, and Creativity. Washington, DC: The National Academies Press. doi: 10.17226/10671.
×

the performing arts, then, for some new-media artists, the answer may be yes. They take information technologies for granted, but their art is not fixated on the computer as a medium, as if it were paint or a violin, or even the sound artist’s turntables or the scenic artist’s optical instruments. As Stephen Wilson’s recent encyclopedic compendium of contemporary intersections between art, science, and technology shows, the information arts range across the life and space sciences, nanotechnology, robotics, and other new materials, as well as IT itself.2 This style of practice does not use technology to create new artworks so much as it uses artistic practice to manage and interpret information at the cusp of technological and scientific research.

This new kind of art and design practice looks increasingly like technical research, but it is done from an artistic or design rather than a scientific perspective—it asks different kinds of questions and uses different kinds of methods to search for answers. Generally speaking, technical research focuses almost exclusively on new technical possibilities: What new things can be done? How can they be done faster or more efficiently? By contrast, artistic and design work tends to focus on the social and cultural meaning of the technology that is under development. This aspect differentiates the approach from that of conventional CS, which does not tend to address explicitly such implications of decisions about system design and implementation, and which may look askance at approaches that have a social science flavor. While a traditional work of art can be thought of as a representation of an artistic concept, the information arts often ask what technologies themselves (perhaps unintentionally) express and how they ought to be reconceived.3

Artists’ questioning can be a powerful, constructive force. In particular, since the mid-19th century artists have often personified the “user to come” for new cultural technologies. Many media technological advances have arisen in the arts and design fields or have been modeled there, a decade or a generation ahead of the industrial-academic curve; see Box 4.1. For Alvy Ray Smith, the prominent computer graphics expert, artists are most valuable as “explorers at the edge of our culture,” and he looks to them to “tell the rest of us what [computation] really is.”4 Thus, the information artist functions as an archetypal knowledge worker: someone able to “penetrate conventional organizations to which their continuing attachment to an ‘external’ knowledge community represents a valuable asset.”5 ITCP

2  

See Stephen Wilson, 2001, Information Arts, MIT Press, Cambridge, Mass. Also see links to online resources at <http://online.sfsu.edu/~infoarts/links/wilson.artlinks2.html>.

3  

The CAT’s MeAoW lecture series at New York University, for example, was framed around such questions; see Box 6.1 for details.

4  

Alvy Ray Smith, 1998, “The Stuff of Dreams (25 Years = 100,000x),” Computer Graphics World 21(7): 27-29.

5  

See Paul A. David and Dominique Foray. 2002. “An Introduction to the Economy of the Knowledge Society,” International Social Science Journal (UNESCO 171) 54:25-37.

Suggested Citation:"4. The Influence of Art and Design on Computer Science Research and Development." National Research Council. 2003. Beyond Productivity: Information Technology, Innovation, and Creativity. Washington, DC: The National Academies Press. doi: 10.17226/10671.
×

BOX 4.1 Far from Unprecedented: Influences of Art and Design on Information Technology

The role of artists in defining technology is well established in history, though this history is often not well-known, as least by modern pundits and chroniclers of information technology (IT). For example, Louis Daguerre, often cited as one of the pioneers of photography, was neither a chemist nor an optics professor; he was a painter of sets for the opera (and the inventor of daguerreotype).1

It is instructive to consider the history of electronic and digital musical instruments. From the point of view of composers, the critical discussions were about timbre, scales, and compositional techniques and influences, thus shaping how electronic and digital music instruments were imagined, used, and developed. The instruments, performances, and means of distribution were all co-developed between artists and engineers.2 The artists’ work was a precondition to commercial exploitation (in the direct and indirect senses).

Such a process is contrary to the traditional historical accounts of engineers (or the military) driving the development of technology and the artists following by finding “creative applications.” Instead, it is often the artists who make the initial investment. The investment of artists is critical to this innovation, as it is artists who are motivated to explore alternatives beyond what has already been framed as acceptable, often long before major commercial applications can be imagined (e.g., the Hammond organ is the direct application of patents filed by a musician before the miniaturization of electronics).

Why are such observations on the way to innovation often omitted from histories of technology evolution? First, art critics often do not “do technology”; art critics and curators, because of their education and expertise, generally do not have much acquaintance with technological concepts, technologies, or theoretical tools for techno-social interaction. Second, because commercial success has become the dominant measure by which to evaluate and justify technological and scientific research, less attention is accorded systematically to non-commercial applications.

1  

See “Daguerre, Louis Jacques Mande” at <http://www.rleggat.com/photohistory/history/daguerr.htm>.

2  

See David Dunn, 1992, “A History of Electronic Music Pioneers,” an essay written for the catalog accompanying the exhibition “Eigenwelt der Apparatewelt: Pioneers of Electronic Art,” Ars Electronica, Linz, Austria, curated by Woody and Steina Vasulka. Also see Trevor Pinch, 2002, Analog Days: The Invention and Impact of the Moog Synthesizer, Harvard University Press, Cambridge, Mass.

makes apparent the value of the artist as mediator—someone who is increasingly intercommunicating—addressing IT-related process and context and expanding beyond the traditional artist’s focus on content.

For an example from the world of design practice, recall the work of Karim Rashid (presented in Chapter 2). The software that controls the variations in each of the napkin rings produced is integral to the creative process. Without such software control, the rings would be identical as the outputs from a mass production process. In both instances (with and without the intervening software), the initial design involves creative practice. However, the introduction of Rashid’s software into the production process offers an additional opportunity for creativity. In this sense, the reach of the information artist extends beyond product design to process design.

The reach of the information artist extends beyond product design to process design.

Suggested Citation:"4. The Influence of Art and Design on Computer Science Research and Development." National Research Council. 2003. Beyond Productivity: Information Technology, Innovation, and Creativity. Washington, DC: The National Academies Press. doi: 10.17226/10671.
×

MODELING DISCIPLINES: FROM MULTIDISCIPLINARY TO TRANSDISCIPLINARY

The relationship between IT and the arts and design as discussed in Chapter 3 can be described using a multidisciplinary model. Each discipline (e.g., architecture) is represented as a circle; the area where the circles overlap is the area of intersection (e.g., IT-enhanced architecture). One implication of this representation is that the non-over-lapping areas do not change. Each discipline provides some piece of its practice or theory that is compatible, useful, or mutually beneficial to other disciplines without any change in the way the discipline itself fundamentally works (see Figure 4.1a).6 This kind of representation conveys how aspects of already-existing genres, methods, or theories are applied in different contexts—in the case of this report, how IT can be applied in the arts and design areas.

But in another possible model (Figure 4.1b), the circles do not intersect but instead share a common frame. Each discipline maintains its own knowledge and methodologies but is fully, not partially, open to the other disciplines.7 In this second model, the disciplines not only apply their methods in a new context but also are receptive to fundamental changes in knowledge and methodology based on their interaction. What is crucial to enable this kind of interaction is the “frame” surrounding the disciplines—a mutual awareness and understanding, especially a historical understanding, of one another and their relationship. The shared frame may be only a transient phenomenon—the disciplines may come into contact, engage in some fruitful exchange, and then continue to develop separately and move apart, as contrasted with the multidisciplinary approach sketched in Figure 4.1a. In transdisciplinary research,8 the point is not just application of given methodologies but also implication—a result of imagining entirely new possibilities for what disciplines can do.

6  

It is worth noting that areas such as “the arts” cannot be reasonably contained within a circle implying that what is “in” can be distinguished easily from what is “out” (or even that an “in” versus “out” categorization is useful). The use of a circle to represent disciplines is, of course, a heuristic for purposes of exposition.

7  

Disciplines can be relatively open or closed in various ways. One way to assess whether a discipline is open is through the use of tools from bibliometrics. One can, for example, employ citation analysis to determine the frequency with which a discipline cites articles in (certain) other disciplines. Those disciplines with a higher percentage of outside-discipline citations could be characterized as more open than disciplines with lower percentages. Standards (and representational structures more generally) can also be examined in their role of affecting openness. Standards (e.g., HTML) can serve as wonderful contributions to work at the intersection of disciplines, but at the same time can be the source of resistance to change in the home discipline.

8  

Margaret A. Somerville and David J. Rapport, eds., 2000, Transdisciplinarity: Recreating Integrated Knowledge, EOLSS, Oxford, U.K.

In transdisciplinary research, the point is not just application of given methodologies but also implication—a result of imagining entirely new possibilities for what disciplines can do.

Suggested Citation:"4. The Influence of Art and Design on Computer Science Research and Development." National Research Council. 2003. Beyond Productivity: Information Technology, Innovation, and Creativity. Washington, DC: The National Academies Press. doi: 10.17226/10671.
×

(a) Multidisciplinary Model: New Context of Application

• Mixes knowledge of disciplines A, B, and C for a common purpose, but each discipline keeps its former shape

• Intersected area creates new context for application of already-existing concepts and methods

—IT as a means to replicate, automate, speed up, and/or reduce the cost of prior analog practices

—Art as post hoc “beautification” of separately conceived functionality styling

(b) Transdisciplinary Model: New Context of Application

Transaction space requires understanding of own and other disciplines

• New common space created specifically from interpenetration of disciplines

• Boundaries are shown as perturbed, and they adjust to accommodate the reflection from other circles

• Intensity of communication between disciplines becomes context for implication— expansion beyond context of immediate application to “anticipatory vision” of future possibilities

• Transaction space may be transient, leaving separate circles reconstituted based on a single transaction—or sustained over time, leading to a durable merging

FIGURE 4.1 Models of the relationship between information technology and the arts and design: Multidisciplinary versus transdisciplinary. SOURCE: Adapted from Margaret A. Somerville and David J. Rapport, eds., 2000, Transdisciplinarity: Recreating Integrated Knowledge, EOLSS, Oxford, U.K, pp. 248-249.

Suggested Citation:"4. The Influence of Art and Design on Computer Science Research and Development." National Research Council. 2003. Beyond Productivity: Information Technology, Innovation, and Creativity. Washington, DC: The National Academies Press. doi: 10.17226/10671.
×

The project on Internet data sonification (introduced in Chapter 2) involving a Lucent Technologies statistician and a media artist/composer illustrates the differences between the two models. Brought together through an ad hoc, short-term artist-in-residency effort co-sponsored by Lucent Technologies and the Brooklyn Academy of Music, neither the scientist nor the artist had a preexisting hypothesis or conception of how to combine musical gesture and statistical modeling of data. The outcome was a prototype listening station that conveys the dynamic properties of Internet communication flows (e.g., newsgroups and chat communities) through sounds: melody, texture, and rhythm. This common object resulted from reciprocal learning about the implications of each field for the other. Beginning with a less joint or problem definition, a multidisciplinary outcome would have been different from this—and arguably less innovative. For example, a composer or a painter might have interpreted the abstract patterns revealed by the data flows; that kind of approach has been seen in art inspired by data visualization. Or the statistician might have proposed to a designer the production of an elegant display of quantitative tables.

The Listening Post research prototype has spawned several sequels, each realized in the separate worlds of the scientist’s and the artist’s ongoing work. Mark Hansen, the scientist, defined applications relevant to the operation of Lucent network-monitoring facilities; Ben Rubin, the artist, continued to present data-driven network sonification in musical and gallery contexts. The team has, in fact, endured longer than the initial pilot.9 Two co-authored papers on the results of the experiment address different specialist readers, accounting for the collaborators’ contribution symmetrically.10 Intellectual property agreements unique to the dynamics of the project were developed after some difficulty; they recognized that the artistic content and the invented intellectual property—together constituting the project outcome—are inextricably merged and are co-owned.11

In the intersection of multiple disciplines described above, the roles of artists and designers and computer scientists are clear-cut. Artists and designers have needs that computer scientists can fulfill. Engaging in a fruitful exchange requires conversations to identify those needs and to determine how computer scientists can best fulfill

9  

Through the sponsorship of the Rockefeller Foundation, their collaboration continues as of this writing: The “Ben Rubin and Mark Hansen: Listening Post” exhibition is running at the Whitney Museum of American Art from December 17, 2002, through March 9, 2003; see <http://www.whitney.org/information/press/87.html>.

10  

Ben Rubin and Mark H. Hansen, 2000, “The Audiences Would Be the Artists and Their Life Would Be the Arts,” IEEE Multimedia 7(2): 6-9; Mark H. Hansen and Ben Rubin, 2001, “Babble Online: Applying Statistics and Design to Sonify the Internet,” pp. 1-15 in Proceedings of the 2001 International Conference on Auditory Display, Espoo, Finland, July 29-August 1.

11  

In 2002, Mark Hansen joined the faculty of the University of California at Los Angeles. Interestingly, Hansen developed two presentations for interviews—one for faculties of science and one for faculties of art and design.

Suggested Citation:"4. The Influence of Art and Design on Computer Science Research and Development." National Research Council. 2003. Beyond Productivity: Information Technology, Innovation, and Creativity. Washington, DC: The National Academies Press. doi: 10.17226/10671.
×

them. In a transdisciplinary situation, however, artists and designers are not clients of computer scientists but instead interact with them as peers. Bringing to the exchange their own disciplinary methodologies and value systems, artists and designers have their own opinions about what research ought to be pursued and how it ought to be done. One result is a fundamental rethinking of how research into information technology might be conceived.

IMPLICATIONS FOR COMPUTER SCIENCE

For computer science work, the advantages of being open to the perspectives of the arts and design disciplines are potentially large. Computer science already has a productive tradition of drawing on other disciplines, from mathematics to physics to cognitive psychology, to advance its own work by exploring new problems and thinking about new potential solutions to those problems.12 Similarly, responding to disciplines from the arts and design worlds opens the possibility of discovering new methodologies for and solutions to problems that, until now, have been beyond the reach of the computer science field to solve or perhaps even articulate. Often the effects of IT research have proved profound (and sometimes unintentionally so),13 and ITCP serves as a way for those with primarily technical interests to communicate with those more interested in the social, cultural, and political aspects of technology.

The perspectives of the information arts are particularly interesting in cases where CS research itself is already moving toward the perspective embodied by art and design practices. One example of such a shift is in the field of human-computer interaction (HCI). In the last 10 years, the field has moved gradually from focusing largely on the hardware and software of human-computer interaction (e.g., development of the mouse and graphical interface) to paying more attention to human psychology (e.g., what mental models of software are constructed by users) and social interaction (e.g., how software can support project collaboration). More recently, HCI has begun to draw more broadly on the social sciences, especially ethnography (the rigorous, qualitative study of human use and contexts of technology), in order to design systems that better fit into the lives of human users. Simultaneously, the connections have deepened between HCI and the design community, which approaches human-computer interaction in more open-ended ways.14 These shifts in HCI as a field bring it closer

12  

See Computer Science and Telecommunications Board, National Research Council, 1992, Computing the Future: A Broader Agenda for Computer Science and Engineering, Juris Hartmanis and Herbert Lin, eds., National Academy Press, Washington, D.C.

13  

For example, it is clear that the developers of Transmission Control Protocol/ Internet Protocol were not looking to challenge the music recording industry, or to enable the wide distribution of technical information on making inexpensive bombs.

14  

See, for example, Terry Winograd, John Bennett, Laura De Young, Peter S. Gordon, and Brad Hartfield, eds., 1996, Bringing Design to Software, ACM Press, New York, N.Y., and Addison-Wesley, Reading, Mass.

One result is a fundamental rethinking of how research into information technology might be conceived.

Suggested Citation:"4. The Influence of Art and Design on Computer Science Research and Development." National Research Council. 2003. Beyond Productivity: Information Technology, Innovation, and Creativity. Washington, DC: The National Academies Press. doi: 10.17226/10671.
×

to the information arts and suggest that there is now a potential for synergy between the two.

One such area where the methodology and attention to social, cultural, and political context typical of the information arts may benefit HCI is the use of technology outside of work contexts. Work applications tend to focus on efficient, problem-solving functionality for which we now have well-understood design and evaluation techniques. Applications for everyday life, however, suggest the importance of aspects that are less understood and are hard to quantify, such as quality of experience, meaningfulness, personal values, identity, and appropriateness to social and cultural context—areas for which the perspectives of the information arts may be particularly appropriate. This concern with the human element can already be seen in consumer electronics, which have a history of drawing on market research and human factors analysis and which often depend on design for competitive advantage; the broader uses of IT envisioned with increases in the embedding of computing and communications components implies a broader and often rather different set of personal and other non-work-focused technologies in the future. HCI researchers are realizing that there is a need to do some fundamental rethinking of HCI methods to understand what these assumptions are, to analyze the extent to which they are applicable to the new contexts of everyday life, and, in cases where they are not applicable, to invent new methodologies that are more appropriate.15 This kind of rethinking is an endeavor for which the information arts can be helpful; concrete examples where interaction may be particularly fruitful are discussed further in the section “Non-utilitarian Evaluation” below.

Similar shifts are occurring in other areas of computer science. In artificial intelligence (AI), for example, there has recently been a focus on lifelike computer characters or believable agents, with a great deal of interest in incorporating approaches from drama and the arts into agent design.16 The development of algorithms for information retrieval on the Web has underscored the need to combine theoretical

15  

See, for example, recent publications of the ACM Press, including the proceedings from the CHI 2002 Workshop on Funology, the HCI 2002 Workshop on Understanding User Experience: Literary Analysis Meets HCI, and the CHI 2003 Workshop on Designing Culturally Situated Technology for the Home; also Bill Gaver and Heather Martin, 2000, “Alternatives: Exploring Information Appliances Through Conceptual Design Proposals,” pp. 209-216 in Proceedings of the CHI 2000 Conference on Human Factors in Computing Systems; ACM Press, New York; Debby Hindus, Scott D. Mainwaring, Nicole Leduc, Anna Elisabeth Hagström, and Oliver Bayley, 2001, “Casablanca: Designing Social Communication Devices for the Home,” pp. 325-332 in Proceedings of CHI ’01, ACM Press, New York; and Jon O’Brien and Tom Rodden, 1997, “Interactive Systems in Domestic Environment,” pp. 247-259 in Proceedings of the 1997 Conference on Designing Interactive Systems, ACM Press, New York.

16  

See, for example, Clark Elliott and Jacek Brzezinski, 1998, “Autonomous Agents as Synthetic Characters,” AI Magazine 19(2): 13-30; and Joseph Bates, 1994, “The Role of Emotion in Believable Agents,” Communications of the ACM 37(7): 122-125. Also see Box 4.2.

Suggested Citation:"4. The Influence of Art and Design on Computer Science Research and Development." National Research Council. 2003. Beyond Productivity: Information Technology, Innovation, and Creativity. Washington, DC: The National Academies Press. doi: 10.17226/10671.
×

computer science with an understanding of the social structure of the Web17 and raises potential connections to the cultural politics of Web information,18 an area in which the information arts are working. For areas like these, in which purely technical solutions do not seem adequate to fully address the problems of interest to computer scientists, interaction and engagement with information arts could be beneficial to computer science.

Because the information arts are inherently transdisciplinary, they hold the possibility of motivating more than just the straightforward use of information arts for computer science’s ends or simple collaboration between information artists and computer scientists. Instead, there can be a mingling and repositioning of each interacting discipline. From the perspective of computer science, this implies a move to more qualitative, rather than quantitative, research methods; a greater incorporation of political, social, and ethical considerations into computer science research; and more focus on intuition and aesthetics.19 Given the movements that have already taken place on the arts and design side, productive cross-fertilization and a broader base for ITCP will depend on the flexibility and openness of individual researchers, research communities, departments, universities, and professional societies—the institutions and organizations that define academic computer science.20 As detailed elsewhere in this report, there is both movement in that direction21 and resistance to such movement.

PROMISING AREAS

During the course of its deliberations, the committee identified a number of promising areas for transdisciplinary work. Several have attracted fairly active interest, whereas others are just emerging. They open the possibility of fruitful discussion and collaboration in these

17  

See J. Kleinberg and S. Lawrence, 2001, “The Structure of the Web,” Science 294(5548): 1849-1850.

18  

Richard Rogers, ed., 2000, Preferred Placement, Jan van Eyck Akademie, Maastricht, The Netherlands.

19  

For a compelling example of how AI research can be rethought to incorporate critical thinking from the humanities, see the discussion of critical technical practices in Philip E. Agre, 1997, Computation and Human Experience, Cambridge University Press, Cambridge, U.K. For a similar integration of critical thinking and HCI, see Paul Dourish, 2001, Where the Action Is, MIT Press, Cambridge, Mass.

20  

See Chapter 6 for discussion.

21  

Sometimes movement may occur in teaching, as opposed to research, the focus of most of the examples in this report. For example, Robert Coover developed “Cave Writing,” a course at Brown University that brought together English students, artists from the Rhode Island School of Design, and computer scientists from the Brown CS department. They explored the creative potential of the Cave, a high-end virtual reality environment for text and related digital media elements of sound and image in virtual space.

Suggested Citation:"4. The Influence of Art and Design on Computer Science Research and Development." National Research Council. 2003. Beyond Productivity: Information Technology, Innovation, and Creativity. Washington, DC: The National Academies Press. doi: 10.17226/10671.
×

areas between computer scientists and artists and designers engaged in computer-science-like work. The discussion below focuses on areas that the information arts are particularly well suited to address based on the following factors: The areas involve the social context or politics of computing; they raise difficult ethical issues that need to be addressed in the context of technical research; they have high public or social impact; and/or they suggest fundamental rethinking of computer science. Although the following compilation is neither comprehensive nor predictive of the most promising areas, it does give an idea of the breadth of possibilities for productive engagement between the information arts and computer science.

MIXED REALITY

Mixed reality is a new, interactive medium in which computing is taken off the desktop or head-mounted display and linked with real-world objects and places to become part of everyday, physical lives. In these approaches, IT development and other creative practices are synergistic. On the one hand, IT provides a new medium for creative expression, opening up a space of possible developments to be explored. Design and media art practice, on the other hand, offer a broader functional and aesthetic perspective. An art and design perspective introduces a cultural awareness that is essential in the development of devices that not only are functional but also contribute to the quality of life in a less direct, but often more profound, way.22

In one classic design, Durrell Bishop’s marble answering machine, each message is represented by a marble (see Figure 4.2).23 When a message is taken, the machine produces a marble.24 The marble can be picked up and put back into the machine in order to play the message. Placed on a matching phone, the marble causes the phone to dial the original caller. Messages are deleted by recycling the marble in the machine. The marble answering machine speaks to humans’ physicality.25

Approaches to mixed reality include tangible media and augmented reality. In tangible media, physical objects like Bishop’s marbles

22  

Considerations beyond the utilitarian are pervasive and dominant in developed economies: The design and production of myriad products and services, from automobiles and clothing to news services and computer systems, incorporate many features that are not strictly necessary from a functional perspective.

23  

Gillian Crampton-Smith, 1995, “The Hand That Rocks the Cradle,” I.D., May/June, pp. 60-65.

24  

The movie Minority Report (produced in 2002 by 20th Century Fox and starring Tom Cruise) employs a similar marble to announce the names of the victim(s) and perpetrator(s) of crimes that will take place in the future.

25  

Dag Svanaes and William Verplank, 2000, “In Search of Metaphors for Tangible User Interfaces,” 2000 Conference on Designing Augmented Reality Environments, Association for Computing Machinery, New York.

For areas in which purely technical solutions do not seem adequate, interaction and engagement with information arts could be beneficial to computer science.

Suggested Citation:"4. The Influence of Art and Design on Computer Science Research and Development." National Research Council. 2003. Beyond Productivity: Information Technology, Innovation, and Creativity. Washington, DC: The National Academies Press. doi: 10.17226/10671.
×

FIGURE 4.2 A conception of Durrell Bishop’s marble answering machine. When a new message is left, the machine deposits a marble in the upper tray (A) for the recipient to find. The marble can then be placed on the lower indentation (B) to play the message. When the message is no longer needed, the recipient recycles it by dropping the marble into the hole (C). Illustration created by Jennifer M. Bishop, Computer Science and Telecommunications Board staff.

have computational properties. Augmented reality is an alternative to virtual reality in which virtual images and data are projected onto and thereby incorporated with the physical world. For example, one can look through augmented-reality binoculars mounted in the atrium lobby of the Center for Art and Media (ZKM) in Karlsruhe, Germany, to see the heart of the building, overlaid with labels explaining what is done on the different floors: reality plus.26 Both augmented reality and tangible media have their roots in Mark Weiser’s vision of ubiquitous computing.27

Technical issues in mixed reality include the maintenance of corre-spondence between real-world and virtual objects, standards for interobject communication, perception (including vision processing, video tracking of objects, plan recognition, and integration of multiple forms of sensory data), spatial reasoning, and learning and adaptation. But designing and constructing mixed-reality devices that are functional, useful, interesting, and desirable are not only technical challenges, but also artistic and practical challenges. University envi-

26  

Work by Jeffrey Shaw; see <http://www.zkm.de>.

27  

A vision in which processors are embedded in everyday objects and networked together would integrate computation invisibly and seamlessly into daily life. See Mark Weiser, 1991, “The Computer for the 21st Century,” Scientific American 265(3): 94-104. Ubiquitous computing is discussed further in the section “Mobile and Ubiquitous Computing,” below.

Suggested Citation:"4. The Influence of Art and Design on Computer Science Research and Development." National Research Council. 2003. Beyond Productivity: Information Technology, Innovation, and Creativity. Washington, DC: The National Academies Press. doi: 10.17226/10671.
×

ronments are one venue where the relevant expertise and aspirations are brought together.

At Georgia Institute of Technology’s Graphics, Visualization, and Usability Center, for example, computer scientist Blair MacIntyre and media theorist Jay David Bolter collaborate on the Sweet Auburn project, cross-informing augmented-reality technology and content development. They are developing applications to support tours of Atlanta’s historic Auburn district, in which “ghosts” from Auburn’s past appear superimposed over the landscape and address users with their stories.28 At the MIT Media Laboratory, Hiroshi Ishii’s Tangible Media group integrates art, design, and human-computer interaction to generate pre-market speculative applications such as music bottles that can be uncorked to release the music inside and “curlybots” that record and play back physical gestures.29 A playful, speculative design approach is taken at the Computer-Related Design program at the Royal College of Art (London), home of Bishop’s marble answering machine and whimsical applications ranging from a telepathic Tamagotchi30 to a bird feeder that use principles of reinforcement learning to teach songbirds new tunes.

COMPUTER GAMES

Computer games, having long ago left their roots as playful experiments for academic computer scientists, are emerging as a contemporary topic of computer science research because advances in many component technologies have driven burgeoning interest in “games” for serious contexts as well as entertainment.31 Thus, for example, in the mid-1990s the Department of Defense (DOD) began to explore prospects for research collaborations among people doing modeling and simulation in defense and entertainment (including games) contexts,32 and the Defense Advanced Research Projects Agency

28  

Blair MacIntyre, Marco Lohse, Jay Bolter, and Emmanuel Moreno, 2001, “Ghosts in the Machine: Integrating 2D Video Actors into a 3D AR System,” pp. 80-83 in Proceedings of 2nd International Symposium on Mixed Reality, Yokohama, Japan, March 14-15.

29  

Hiroshi Ishii and Brygg Ullmer, 1997, “Tangible Bits: Seamless Interfaces Between People, Bits and Atoms,” pp. 234-241 in Proceedings of the 1997 Conference on Computer-Human Interaction, ACM Press, New York.

30  

A Tamagotchi (pronounced “tom-ah-GOT-chee”) is a relatively inexpensive toy containing a small liquid-crystal display, a few touch-sensitive user controls, and a program in which the image of a small creature is visible. Users can see the creature mature or, if insufficient attention is paid, see the creature die. Tamagotchi comes from the Japanese terms “tamago” meaning “egg” and “chi” as a term of endearment; it means, approximately, “lovable egg.” Derived from <http://whatis.techtarget.com/definition/0,,sid9_gci213089,00.html>. Also see Anthony Dunne, 1999, Hertzian Tales: Electronic Products, Aesthetic Experience & Critical Design, Art Books International Ltd., London.

31  

The popular appeal and success of computer games are discussed in Chapter 2.

32  

See Computer Science and Telecommunications Board, National Research Council, 1997, Modeling and Simulation: Linking Entertainment and Defense, National Academy Press, Washington, D.C.

Suggested Citation:"4. The Influence of Art and Design on Computer Science Research and Development." National Research Council. 2003. Beyond Productivity: Information Technology, Innovation, and Creativity. Washington, DC: The National Academies Press. doi: 10.17226/10671.
×

has begun to explore the potential of games for decision support. The DOD exploration gave rise to an Army-funded center at the University of Southern California, the Institute for Creative Technology; related work had already begun at the Naval Postgraduate School in Monterey, California.

Computer games offer a unique playground for serious research, not only because of the underlying allure of fun and competition, but also because important new questions arise. For example, what is a body, a surface (when infinitely malleable), or a space? How does one deal with a changing sense of time given that one can go back to a saved game? How then does one change the way one plays? How does one convey the essence of person despite screen form—gestures, and so on—varying? Most interestingly, designers of massively multiplayer online games are grappling, with a large degree of success, with the social, political, and aesthetic issues inherent in virtual worlds. What is the social contract between participants, and between the participant and the designer? What are the consequences of conflicts in the virtual world, and to what degree should those consequences be determined by the online population, versus the administration? How should people deal with the distribution of authorship in an environment where narratives are participatory and emergent? How does one foster organic, self-organizing social structures in a virtual world? How does a designer make places people want not only to visit, but also to inhabit for hundreds or thousands of hours over the course of several years? These questions raise various issues for a number of computer science fields, including information retrieval, database management, and computer graphics, to name a few—though such questions are not purely CS ones, but rather questions that are truly transdisciplinary. There is evidence that CS is beginning to address some of these questions (e.g., see the special issue “Game Engines in Scientific Research” in the Communications of the ACM, January 2002).33

NARRATIVE INTELLIGENCE

In the early 1990s, a group of graduate students at the MIT Media Lab formed a new reading group, which they called narrative intelligence (NI).34 The group explored issues at the intersection of narrative and both human intelligence and AI, seeking to develop a dialogue between new computational concepts and technologies and the insights of literary theories such as poststructuralism and semiotics. The group came together with an understanding of, and the desire to

33  

One reviewer observed that the CS research agenda was being only modestly influenced as of November 2002.

34  

Marc Davis and Michael Travers, 1999, “A Brief Overview of the Narrative Intelligence Reading Group,” pp. 11-16 in Proceedings of the 1999 AAAI Symposium on Narrative Intelligence, Michael Mateas and Phoebe Sengers, eds., AAAI Press, Menlo Park, Calif.

Suggested Citation:"4. The Influence of Art and Design on Computer Science Research and Development." National Research Council. 2003. Beyond Productivity: Information Technology, Innovation, and Creativity. Washington, DC: The National Academies Press. doi: 10.17226/10671.
×

reconcile, the contradictions and incompatibilities between these two world views: AI technology focused by and large on formal, logical representation and objectivity, whereas the analytical tools provided by new literary theories focused on subjectivity, multiplicity, and the limitations of formalism. The pragmatics of negotiating the differences between these world views led to a creative foment. The group flourished, exploring issues in the philosophy of mind, media theory, HCI, psychology, social computing, constructionism, and AI, developing theories and applications in all these areas, influencing the direction of the doctoral program at the Media Lab, and connecting to a wider network of researchers who joined in the group’s discussions over e-mail. Narrative intelligence as a field was born.

NI research obviously incorporates influences from a variety of fields. Artificial intelligence, with tools to model human emotion, personality, and narrative abilities, provides a framework from which much of the research grows. Psychology, especially narrative psychology, generates explanations of the human ability to understand the world through narrative, creating a basis for systems that model or support this ability. Art research raises new questions about the nature of narrative representation, keeping the concept of narrative fresh. Cultural studies analyze hidden cultural narratives, including the stories AI researchers tell through their research. Literary studies examine the nature of narrative in traditional and interactive forms. Drama provides understanding of the real-time performance of narrative. This emphasis on mixing technology development with artistic and humanistic perspectives is unusual in AI. It has supported the generation of new research fields within AI, such as lifelike interactive computer characters, as well as an increase in cross-disciplinary engagement between AI and other fields.35

At the same time, narrative trends took on importance in related fields. The concept of supporting human narrative understanding through the interface of human and computer began to gain ground in the field of HCI. Work in media studies on hypertext and interactive fiction was inspiring a generation of systems that support narrative in new ways. Within AI, this interest began to spur research in AI for interactive fiction and entertainment,36 including interactive computer

35  

Narrative as a topic of research is not new to AI. In the 1970s and early 1980s, there was substantial interest in modeling story understanding and story generation, particularly by Roger Schank’s research group at Yale University. Programs developed by the group—which were able to generate stories and answer questions about them, albeit in limited ways and domains—illustrated theories of human understanding and the structure of knowledge in the mind. Massive, unwieldy, and hard to extend, these systems ran into trouble during the “AI winter” of the 1980s. Researchers, seeking to combat the image of AI as never living up to its inflated claims, favored clearly defined problems with easily measurable outcomes—a situation that is not conducive to this kind of creative work—and, therefore, narrative fell out of favor for a decade as a research topic in AI.

36  

Joseph Bates, 1992, “Virtual Reality, Art, and Entertainment,” Presence: The Journal of Teleoperators and Virtual Environments 1(1): 133-138.

Suggested Citation:"4. The Influence of Art and Design on Computer Science Research and Development." National Research Council. 2003. Beyond Productivity: Information Technology, Innovation, and Creativity. Washington, DC: The National Academies Press. doi: 10.17226/10671.
×

BOX 4.2 Virtual Characters—Improvisational Actors

The research, development, and commercialization of a new class of intelligent-agent applications that reflect a philosophy of learning through play is the focus of Extempo Systems. Software-driven virtual characters, which Extempo Systems’ founder Barbara Hayes-Roth1 calls improvisational actors, can take the form of a human, an animal, or just about anything the imagination can dream up, to interact directly with people visiting a Web site. What makes them different from other types of intelligent agents is their ability to improvise stories. According to Hayes-Roth, this design allows the actors to achieve their goals while providing engaging conversation for many different Web site visitors. One of Extempo’s creations, a dog named Jack, seeks to achieve six goals when people interact with him: get them involved, guide them to target information, gather customer data, personalize the experience, delight them, and build site loyalty. Jack accomplishes his goals by improvising appropriately in response to a situation. For example, as a visitor enters a site, Jack says what he can do for the person. If the visitor stops “talking,” Jack is programmed with a few things to say based on the content of the conversation thus far. Like a human, Jack is designed to have a certain protocol involving turn taking and interruption. This allows him to have characteristics of intelligent conversation, such as making functional transformations and analogizing between topics.

1  

See <http://www.extempo.com> for additional information about Extempo Systems and the work of Barbara Hayes-Roth, who was a briefer at the committee’s January 2001 meeting held at Stanford University.

characters and interactive plots. Many of these research areas explicitly draw on the arts and drama as a source of inspiration. With the growth of the computer game industry has come an interest in new game forms that support narrative in more complex and interesting ways than a stereotypical shoot-and-kill form.

Research in NI is flourishing, with applications in a variety of areas. Narrative interfaces explore possibilities for making interfaces more usable by incorporating elements of story, for example by embodying interaction in a storytelling character. Artificial agents can themselves be designed to use narrative, as humans do, to make sense of the world and each other (see Box 4.2). Researchers are developing systems to support human storytelling, as in the case of plush toys that children can program to tell their stories to families and friends.37 Databases of stories allow people to search for and share stories pertinent to their experiences.38 Stories can be automatically generated, perhaps in response to input from human users. Interactive digital video allows video sequences to be generated interactively, telling

37  

Marina Umaschi, 1997, “Soft Toys with Computer Hearts: Building Personal Storytelling Environments,” pp. 20-21 in CHI ‘97 Proceedings, ACM Press, New York.

38  

See Justin Cassell and Jennifer Smith, 1999, “The Victorian Laptop,” pp. 72-78 in Narrative Intelligence: Papers from the 1999 Fall Symposium, Technical Report FS-99-01, Michael Mateas and Phoebe Sengers, eds., AAAI Press, Menlo Park, Calif.

Suggested Citation:"4. The Influence of Art and Design on Computer Science Research and Development." National Research Council. 2003. Beyond Productivity: Information Technology, Innovation, and Creativity. Washington, DC: The National Academies Press. doi: 10.17226/10671.
×

interactive stories.39 The field of interactive fiction and drama has exploded,40 including the subfield of interactive computer characters, or characters with emotion and personality who respond to human users in the context of a story.41 A complementary area of narrative intelligence studies the stories that AI researchers themselves tell about what they are doing.42 Sometimes, analysis of these stories can lead to new forms of AI technology by building on alternative stories.43

In this explosion of research, the interdisciplinary engagement begun by the NI group at the Media Lab remains present—in work taking place in traditional computer science departments, in cross-disciplinary arenas like the Media Lab, in humanities and arts departments that incorporate new media such as Georgia Institute of Technology’s School of Literature, Communication, and Culture,44 and in the computer game industry.

NON-UTILITARIAN EVALUATION

As discussed above, artists traditionally use evaluation techniques that differ radically from those of computer scientists, with little interest in formal user studies and more interest in social impact, cultural meaning, and the potential political implications of a technology. They seek to provoke as well as to understand the user. There is an opportunity to develop hybrid evaluation methodologies to combine the broader concerns of artists with the narrower and more structured methodologies of HCI. For example, Angela Garabet, Steve Mann, and James Fung use strategies that are open-ended and interpretive45 to evaluate users’ reactions to wearable computing designs. Interestingly, they demonstrate that users are more open to and accepting of new technology that is presented as the product of a commercial venture rather than as art. Jonas Lundberg and colleagues uninten

39  

See Glorianna Davenport and Michael Murtaugh, 1997, “Autonomist Storyteller Systems and the Shifting Sands of Story,” IBM Systems Journal 46(3): 446-456.

40  

For example, see Peter Weyhrauch, 1997, Guiding Interactive Drama, Ph.D. Thesis, School of Computer Science, Carnegie Mellon University, Technical Report CMU-CS-97-109, Pittsburgh, Pa.

41  

See Bruce Mitchell Blumberg, 1996, Old Tricks, New Dogs: Ethology and Interactive Creatures, Ph.D. Thesis, MIT Media Laboratory, Cambridge, Mass.

42  

See N. Katherine Hayles, 1999, How We Became Posthuman: Virtual Bodies in Cybernetics, Literature, and Informatics, University of Chicago Press, Chicago.

43  

See Philip E. Agre, 1997, Computation and Human Experience, Cambridge University Press, Cambridge, U.K.; also Phoebe Sengers, 1998, Anti-boxology: Agent Design in Cultural Context, Ph.D. Thesis, School of Computer Science, Carnegie Mellon University, Technical Report CMU-CS-98-151, Pittsburgh, Pa.

44  

Described in Chapter 6.

45  

For example, because evaluators are trying to understand how people react to a system, users might not be told about the purpose or operation of a system. See Angela Garabet, Steve Mann, and James Fung, 2002, “Exploring Design Through Wearable Computing Art(ifacts),”Computer-Human Interaction 2002, Interactive Poster: Fun, pp. 634-635.

Suggested Citation:"4. The Influence of Art and Design on Computer Science Research and Development." National Research Council. 2003. Beyond Productivity: Information Technology, Innovation, and Creativity. Washington, DC: The National Academies Press. doi: 10.17226/10671.
×

tionally achieved similar results in their explorations of a provocative technology, a refrigerator that videotaped its users, ostensibly allowing those who shared the refrigerator to find out if someone had stolen their food. Although the goal was to confront users with negative aspects of technology, users who saw the “product” demonstrated in an ostensibly commercial presentation were surprisingly enthusiastic.46 Such results may motivate some artists to be more interested in collaborations with commercial objectives than they might otherwise be.

Evaluation techniques drawing on both HCI and arts traditions could rigorously examine not only the usability and utility of software and electronic products, but also the meanings they may take on in users’ everyday lives, the background cultural assumptions that underlie them (for example, the assumptions designers make about what users are like), and their potential impact on current cultural issues and debates, such as intellectual property issues. At the same time, standard HCI techniques appropriately adapted to the goals of artists (often far removed from issues of usefulness and efficiency that current techniques can address) may help improve the sometimes opaque design of interactive artwork. To achieve these goals will likely require a fundamental rethinking of the notion of user tests, as well as other evaluations. In an early example of what such work might look like, artist-designers Anthony Dunne and Fiona Raby evaluated the Placebo project, electronically enhanced furniture that makes users aware of activity in the electromagnetic spectrum, through open-ended interviews with users combined with photographic portraits of users with their devices.47 Such techniques allow designers to do evaluation in a form that is to some extent recognizable and understandable to HCI practitioners, while exploring issues that matter to artists, such as the subjective nature of user experience, the stories that give devices not only functionality but also meaning in human context, and the messages that information technologies intentionally or unintentionally communicate to users. There is a need to develop a repertoire of evaluation techniques appropriate for these more open-ended questions that is as wide and deep as that already available for the relatively well defined problems of usability and efficiency.

EXPERIMENTAL CONSUMER PRODUCT DESIGN

Computing is creating new challenges as it moves into everyday life. The impact of IT on everyday culture is felt particularly strongly through electronic consumer products such as handheld computers,

46  

Jonas Lundberg, Aseel Ibrahim, David Jönsson, Sinna Lindquist, and Pernilla Qvarfordt, 2002, “The Snatcher Catcher: An Interactive Refrigerator,” Short Paper, NordiChi 2002, pp. 209-211.

47  

Anthony Dunne and Fiona Raby, 2001, Design Noir: The Secret Life of Electronic Objects, August/Birkhaeuser, Basel, Switzerland.

Suggested Citation:"4. The Influence of Art and Design on Computer Science Research and Development." National Research Council. 2003. Beyond Productivity: Information Technology, Innovation, and Creativity. Washington, DC: The National Academies Press. doi: 10.17226/10671.
×

music storage and playback devices, and electronic toys. Designers Fiona Raby and Anthony Dunne have argued that such consumer products are currently designed much the same way as Hollywood movies: They are generally uncontroversial, focused on socially acceptable needs, and broadly marketed, and they serve an optimistically idealized lifestyle. In the analogy to film, they note that the alternatives to mainstream Hollywood film—such as film noir, experimental film, and independent cinema—consistently develop techniques of narrative and visual style that are later adopted by Hollywood, thus in effect serving an R&D role.48 Likewise, experimental designers often develop ideas that, while often not immediately marketed, influence and eventually help redirect contemporary design practices in marketed products.

Experimental designers explore a range of issues and ideas that are often different from those of individuals working in specific product fields who are more constrained by the demands of the market. Their work often explores issues at the intersection of product design and social issues. For example, the 2002 show of the Interaction Design program at the Royal College of Art included Pedro Sepulveda’s architectural designs responding to fears and anxieties about cell phone radiation. There is tremendous potential for ITCP not only in reconfiguring existing consumer electronic applications, but also in imagining and building prototypes for new applications and markets.

MOBILE AND UBIQUITOUS COMPUTING

As hardware components become smaller, faster, and cheaper, IT is being embedded into more and more physical devices, linked together through (often wireless) networks. Networked systems of embedded computers (EmNets) will be largely invisible but extremely powerful, allowing information to be collected, shared, and processed in new ways. EmNets promise significant changes in environmental and personal monitoring as well as scientific research. Thousands or millions of sensors could monitor the environment, the battlefield, the home, the office, or the factory floor; smart space containing intelligent surfaces and appliances would provide access to computational resources.49

However, product development often follows the path of least resistance, resulting in products that are technically new but do not take full advantage of the broad conceptual design space that is opened up by mobile and ubiquitous technologies. For instance, it is not uncommon to simply replace discrete analog or digital electronics with an inexpensive microcontroller, or to imagine faster or smaller

48  

Dunne and Raby, 2001, Design Noir.

49  

This paragraph is drawn from Computer Science and Telecommunications Board, National Research Council, 2001, Embedded, Everywhere: A Research Agenda for Networked Systems of Embedded Computers, National Academy Press, Washington, D.C.

Suggested Citation:"4. The Influence of Art and Design on Computer Science Research and Development." National Research Council. 2003. Beyond Productivity: Information Technology, Innovation, and Creativity. Washington, DC: The National Academies Press. doi: 10.17226/10671.
×

versions of existing applications. These approaches reflect an internal technical logic and a safe approach to introducing new products to consumers in a competitive marketplace. As a result, they can ignore some of the larger challenges in designing devices that could and should meaningfully support everyday quality of life.

The views of social scientists, designers, and artists are needed to address the potential implications of EmNets. Imagine living in a world where it is impossible for anyone to get lost anywhere. Cars, wireless devices, laptops, TVs, and even articles of clothing will always be able to tell you and others exactly where you are. Imagine also living in a world where devices recognize a person’s speech, emotions (through physical expressions and tone of voice), and likes and dislikes, and respond accordingly. Now, many who use e-mail and the Web enjoy relationships that involve real, but digitized, humans. Will technology make so much available without complaint that the need for interpersonal relations dissolves? Will that relationship with technology generate an illusion of satisfaction, knowledge, creativity, and life? These are the kinds of questions that influence IT research, and they are also the kinds of questions that inspire artistic exploration via ITCP50—both activities where government-funded academic research or philanthropically supported arts-based activity can explore ideas not likely to flow from conventional commercial efforts.51

The development of smart appliances for the home provides an interesting case study. Many of the gadgets being developed today— from refrigerators that can determine when to order more milk and scales that monitor users’ weight gain and suggest low-calorie recipes to home entertainment systems that remember the preferred settings of different users in the home—are obvious extensions of already-existing technologies. The approach of the information arts, in which technical questions are seen as interrelated with social and cultural questions, lends itself well to a more fundamental shift in the design of smart appliances to support not only new technologies but also new, better, and/or more interesting ways of living in the home. The Domestic Environments project of the Equator research collaboration,52 funded by the U.K. Engineering and Physical Sciences Research Council, is one model of how computer scientists collaborating with artist-designers and social scientists can develop appliances that are interesting both technically and socially. One of the appliances is a “drift

50  

Although this chapter focuses on how ITCP, or the information arts, might influence the discipline of computer science, clearly ITCP can (and does) influence the agenda for artists and designers as well. In this sense, IT does much more than provide tools to the arts and design disciplines.

51  

As a comparison, consider the challenge of developing new kinds of content and applications for broadband networks. As CSTB has observed previously, there is an important role for academic research. See Computer Science and Telecommunications Board, National Research Council, 2002, Broadband: Bringing Home the Bits, National Academy Press, Washington, D.C.

52  

See <http://www.equator.ac.uk/>.

Suggested Citation:"4. The Influence of Art and Design on Computer Science Research and Development." National Research Council. 2003. Beyond Productivity: Information Technology, Innovation, and Creativity. Washington, DC: The National Academies Press. doi: 10.17226/10671.
×

table” designed to promote daydreaming and reflection. A “window” built into the table shows images of the British countryside. As objects are placed on the table, the window begins to drift slowly over the countryside.

The focus on security resulting from the September 11, 2001, terrorist attacks may well be a theme worthy of exploration. As technologists push ahead an agenda for more robust computer security, the information arts would promote an agenda of how to improve security with the least harm to society (or even allowing for the possibility of improved security and a net positive gain to quality of life).53 Additional areas of interest include work inspired by the implications of intellectual property law and policy for the digital environment (see Chapter 7 for a discussion of digital copyright) and bioinformatics.

CONCLUSION

Two different kinds of intersection between IT and creative practices are presented in Chapters 3 and 4. On the one hand, Chapter 3 looks at the use of information technology as a medium for art and design practices, suggesting that computer science can support ITCP. On the other, Chapter 4 looks at ways in which art practice and design and computer science can become fused, leading to new fundamental insights into the nature of computer science itself. In practice, these two kinds of intersection are not disjoint. For example, imagine the development of new word processors with input from professional writers (see Box 4.3).

In policy circles, a vigorous debate has been taking place in recent years about whether knowledge production at large is shifting from discipline-bound, strongly bounded, and relatively stable models to transdisciplinary, loosely coupled, and transient ones.54 There is little to be gained by preferring either multidisciplinary or transdisciplinary exchanges; both have their place and are capable of generating useful and interesting results.

53  

Of course, new security sensitivities could have the side effect of complicating ITCP activities that seem to take advantage of weaknesses in order to manipulate conventional IT.

54  

David et al. (1999) attacks Gibbons et al. (1994); Gibbons et al. (2001) responds. Paul David, Dominique Foray, and W. Edward Steinmueller, 1999, “The Research Network and the New Economics of Science: From Metaphors to Organizational Behaviors,” in The Organization of Economic Innovation in Europe, A. Gambardella and F. Malerba, eds., Cambridge University Press, Cambridge, U.K., and New York; Michael Gibbons, Camille Limoges, H. Nowotny, S. Schwartzman, P. Scott, and M. Trow, 1994, The New Production of Knowledge. The Dynamics of Science and Research in Contemporary Societies, Sage, London; Michael Gibbons, Helga Nowotny, and Peter Scott, 2001, Rethinking Science: Knowledge and the Public in an Age of Uncertainty, Blackwell, Malden, Mass.

There is little to be gained by preferring either multidisciplinary or transdisciplinary exchanges; both are capable of generating useful and interesting results.

Suggested Citation:"4. The Influence of Art and Design on Computer Science Research and Development." National Research Council. 2003. Beyond Productivity: Information Technology, Innovation, and Creativity. Washington, DC: The National Academies Press. doi: 10.17226/10671.
×

BOX 4.3 Information Technology and Word Processing

Consider IT tools that support writing, such as the word processor. Most word processors are based on an underlying representation of text as a long string of characters, each having some properties affecting its surface appearance. This was not a deeply thought out metaphor for the process of writing; rather it evolved through generations of automation tools—the word processor as a better typewriter, or a software tool for automating typesetting. Some word processors evolved to have a rudimentary structural representation of writing—outlines, for example—but the basic representation shines through even style sheets and wizards. An apparent benefit of this approach is that it supports all kinds of writing equally well: IT does not attempt to reinforce one writing style or approach to composition in favor of another (ignoring, for the moment, the effects of grammar checkers and other add-ons that attempt to critique the actual writing). However, there is little doubt that the ease with which revisions can be made, ranging from typographical corrections to particular large-scale structural changes, has influenced the approach one takes to writing. This design reflects choices by software developers to structure writing as a simple string of characters, provide context-blind editing operations such as copy and paste, and let users work through the higher-level concepts and details for themselves.

As an example of a realization of a completely different set of design choices, consider Dramatica Pro— a popular tool for developing screenplays.1 Superficially, it is still a tool for supporting writing. However, it uses a domain model for screenwriting, including characters, themes, scenes, plot progression, character development, and so on. It claims 32,768 “story forms,” allows four different viewpoints on the story being told (“throughlines”), and contains models of how plots develop, how character traits relate to one another, and how characters can respond to challenges. An intended side effect of this approach is that it forces the user to consider certain aspects of structure, character, plot, and theme, claiming the benefits of “expanding your creative potential.” Without passing judgment on this particular tool, it is fair to note that its design reflects different choices—an attempt to deeply embody domain knowledge, a focus on supporting higher-level details—and, in fact, provides very limited editing operations such as copy and paste and virtually ignores issues related to the surface appearance of text. It is intended to address a much narrower set of tasks than a general-purpose word processing tool and, as such, would be of little use in writing a report such as this one. Could a writer create a new kind of screenplay for a radically new kind of film using this tool, or would such a creation be outside its scope? Is it conceivable that a writer could push against the boundaries created by such a tool, abuse its constructs in various ways, and produce something surprising to the tools designers? Does having these boundaries against which to push perhaps expand creative potential even more than having structures within which to work?2

Not enough is known about the deep impact of such design choices on users. However, it could prove very useful to explore, as an example, different kinds of word processors that could be designed, for instance, based on radically different notions of what writing is about, or on some deep criticism of the underlying assumptions embodied in typical word processors from the point of view of writers. The result could be a meaningful intellectual shift in computer science, as well as a tool that is much more useful for writers— and possibly for the rest of us as well.

1  

This discussion is based on version 4.0 of Dramatica. See <http://www.dramatica.com/> for details.

2  

Dramatica is intended as a tool for developing story lines, not for producing the screenplay document. However, it does interface with a more traditionally styled word processor, tailored to the unique needs of producing screenplays. This is another example of the kind of choices made by a designer. Further research into innovative tools for supporting creative writing might make it possible to integrate both metaphors within a single tool.

Suggested Citation:"4. The Influence of Art and Design on Computer Science Research and Development." National Research Council. 2003. Beyond Productivity: Information Technology, Innovation, and Creativity. Washington, DC: The National Academies Press. doi: 10.17226/10671.
×

But both the multidisciplinary and the transdisciplinary models make clear that there is a continuing need to maintain the integrity of the traditional disciplines, both in the arts and in the sciences. Without a disciplinary frame, the richness of disciplinary practices, methodologies, and concepts can become lost, leaving an oversimplified cross-disciplinary knowledge domain. This danger exists when any practice is digitized in the absence of an appropriate model, as for example in arts education when young people have become wedded to the prescripted options of packaged applications and are only capable of creating PhotoShop art. What Paul David and his co-authors fear would become “cut-price research motels” in scientific research55 corresponds closely to the degeneration of artistic quality that is possible where electronic art forms (or media art or the modish “new media”) have been cut loose from their deep connections to older and richer art practices.

Finally, it should be noted that the transient, loose coupling of transdisciplinary creativity runs an ever-present risk of premature bureaucratization.56 A single successful outcome is a necessary but by no means sufficient reason to continue cross-disciplinary work in the same vein. In some cases, the outcome of a rich experimental device is best evaluated and further developed in the separate but transformed disciplines that contributed to it. In other cases, however, the committee has found persuasive evidence of the need for the sustained bridging of disciplines, involving the development of both individual practices and a community of researchers in the cross-disciplinary area with correspondingly innovative institutional structures (and these are discussed in Chapter 5).

55  

David et al., 1999, “The Research Network and the New Economics of Science,” p. 334.

56  

As one reviewer notes, there is much to learn from history. See Carolyn Marvin, 1998, When Old Technologies Were New, Oxford University Press; Brian Winston, 1998, Media Technology and Society, Routledge, London; and Paul N. Edwards, 1996, The Closed World: Computers and the Politics of Discourse in Cold War America, MIT Press, Cambridge, Mass. But much of ITCP history is undocumented or otherwise not very accessible, and therefore, it is unteachable in an organized way. As a result, practitioners are unaware of precedents and relevant prior research, and they constantly reinvent the wheel.

Suggested Citation:"4. The Influence of Art and Design on Computer Science Research and Development." National Research Council. 2003. Beyond Productivity: Information Technology, Innovation, and Creativity. Washington, DC: The National Academies Press. doi: 10.17226/10671.
×
Page 96
Suggested Citation:"4. The Influence of Art and Design on Computer Science Research and Development." National Research Council. 2003. Beyond Productivity: Information Technology, Innovation, and Creativity. Washington, DC: The National Academies Press. doi: 10.17226/10671.
×
Page 97
Suggested Citation:"4. The Influence of Art and Design on Computer Science Research and Development." National Research Council. 2003. Beyond Productivity: Information Technology, Innovation, and Creativity. Washington, DC: The National Academies Press. doi: 10.17226/10671.
×
Page 98
Suggested Citation:"4. The Influence of Art and Design on Computer Science Research and Development." National Research Council. 2003. Beyond Productivity: Information Technology, Innovation, and Creativity. Washington, DC: The National Academies Press. doi: 10.17226/10671.
×
Page 99
Suggested Citation:"4. The Influence of Art and Design on Computer Science Research and Development." National Research Council. 2003. Beyond Productivity: Information Technology, Innovation, and Creativity. Washington, DC: The National Academies Press. doi: 10.17226/10671.
×
Page 100
Suggested Citation:"4. The Influence of Art and Design on Computer Science Research and Development." National Research Council. 2003. Beyond Productivity: Information Technology, Innovation, and Creativity. Washington, DC: The National Academies Press. doi: 10.17226/10671.
×
Page 101
Suggested Citation:"4. The Influence of Art and Design on Computer Science Research and Development." National Research Council. 2003. Beyond Productivity: Information Technology, Innovation, and Creativity. Washington, DC: The National Academies Press. doi: 10.17226/10671.
×
Page 102
Suggested Citation:"4. The Influence of Art and Design on Computer Science Research and Development." National Research Council. 2003. Beyond Productivity: Information Technology, Innovation, and Creativity. Washington, DC: The National Academies Press. doi: 10.17226/10671.
×
Page 103
Suggested Citation:"4. The Influence of Art and Design on Computer Science Research and Development." National Research Council. 2003. Beyond Productivity: Information Technology, Innovation, and Creativity. Washington, DC: The National Academies Press. doi: 10.17226/10671.
×
Page 104
Suggested Citation:"4. The Influence of Art and Design on Computer Science Research and Development." National Research Council. 2003. Beyond Productivity: Information Technology, Innovation, and Creativity. Washington, DC: The National Academies Press. doi: 10.17226/10671.
×
Page 105
Suggested Citation:"4. The Influence of Art and Design on Computer Science Research and Development." National Research Council. 2003. Beyond Productivity: Information Technology, Innovation, and Creativity. Washington, DC: The National Academies Press. doi: 10.17226/10671.
×
Page 106
Suggested Citation:"4. The Influence of Art and Design on Computer Science Research and Development." National Research Council. 2003. Beyond Productivity: Information Technology, Innovation, and Creativity. Washington, DC: The National Academies Press. doi: 10.17226/10671.
×
Page 107
Suggested Citation:"4. The Influence of Art and Design on Computer Science Research and Development." National Research Council. 2003. Beyond Productivity: Information Technology, Innovation, and Creativity. Washington, DC: The National Academies Press. doi: 10.17226/10671.
×
Page 108
Suggested Citation:"4. The Influence of Art and Design on Computer Science Research and Development." National Research Council. 2003. Beyond Productivity: Information Technology, Innovation, and Creativity. Washington, DC: The National Academies Press. doi: 10.17226/10671.
×
Page 109
Suggested Citation:"4. The Influence of Art and Design on Computer Science Research and Development." National Research Council. 2003. Beyond Productivity: Information Technology, Innovation, and Creativity. Washington, DC: The National Academies Press. doi: 10.17226/10671.
×
Page 110
Suggested Citation:"4. The Influence of Art and Design on Computer Science Research and Development." National Research Council. 2003. Beyond Productivity: Information Technology, Innovation, and Creativity. Washington, DC: The National Academies Press. doi: 10.17226/10671.
×
Page 111
Suggested Citation:"4. The Influence of Art and Design on Computer Science Research and Development." National Research Council. 2003. Beyond Productivity: Information Technology, Innovation, and Creativity. Washington, DC: The National Academies Press. doi: 10.17226/10671.
×
Page 112
Suggested Citation:"4. The Influence of Art and Design on Computer Science Research and Development." National Research Council. 2003. Beyond Productivity: Information Technology, Innovation, and Creativity. Washington, DC: The National Academies Press. doi: 10.17226/10671.
×
Page 113
Suggested Citation:"4. The Influence of Art and Design on Computer Science Research and Development." National Research Council. 2003. Beyond Productivity: Information Technology, Innovation, and Creativity. Washington, DC: The National Academies Press. doi: 10.17226/10671.
×
Page 114
Suggested Citation:"4. The Influence of Art and Design on Computer Science Research and Development." National Research Council. 2003. Beyond Productivity: Information Technology, Innovation, and Creativity. Washington, DC: The National Academies Press. doi: 10.17226/10671.
×
Page 115
Suggested Citation:"4. The Influence of Art and Design on Computer Science Research and Development." National Research Council. 2003. Beyond Productivity: Information Technology, Innovation, and Creativity. Washington, DC: The National Academies Press. doi: 10.17226/10671.
×
Page 116
Suggested Citation:"4. The Influence of Art and Design on Computer Science Research and Development." National Research Council. 2003. Beyond Productivity: Information Technology, Innovation, and Creativity. Washington, DC: The National Academies Press. doi: 10.17226/10671.
×
Page 117
Next: 5. Venues for Information Technology and Creative Practices »
Beyond Productivity: Information Technology, Innovation, and Creativity Get This Book
×
Buy Paperback | $48.00 Buy Ebook | $38.99
MyNAP members save 10% online.
Login or Register to save!
Download Free PDF

Computer science has drawn from and contributed to many disciplines and practices since it emerged as a field in the middle of the 20th century. Those interactions, in turn, have contributed to the evolution of information technology – new forms of computing and communications, and new applications – that continue to develop from the creative interactions between computer science and other fields.

Beyond Productivity argues that, at the beginning of the 21st century, information technology (IT) is forming a powerful alliance with creative practices in the arts and design to establish the exciting new, domain of information technology and creative practices—ITCP. There are major benefits to be gained from encouraging, supporting, and strategically investing in this domain.

  1. ×

    Welcome to OpenBook!

    You're looking at OpenBook, NAP.edu's online reading room since 1999. Based on feedback from you, our users, we've made some improvements that make it easier than ever to read thousands of publications on our website.

    Do you want to take a quick tour of the OpenBook's features?

    No Thanks Take a Tour »
  2. ×

    Show this book's table of contents, where you can jump to any chapter by name.

    « Back Next »
  3. ×

    ...or use these buttons to go back to the previous chapter or skip to the next one.

    « Back Next »
  4. ×

    Jump up to the previous page or down to the next one. Also, you can type in a page number and press Enter to go directly to that page in the book.

    « Back Next »
  5. ×

    Switch between the Original Pages, where you can read the report as it appeared in print, and Text Pages for the web version, where you can highlight and search the text.

    « Back Next »
  6. ×

    To search the entire text of this book, type in your search term here and press Enter.

    « Back Next »
  7. ×

    Share a link to this book page on your preferred social network or via email.

    « Back Next »
  8. ×

    View our suggested citation for this chapter.

    « Back Next »
  9. ×

    Ready to take your reading offline? Click here to buy this book in print or download it as a free PDF, if available.

    « Back Next »
Stay Connected!