National Academies Press: OpenBook

Beyond Productivity: Information Technology, Innovation, and Creativity (2003)

Chapter: 3. Advancing Creative Practices through Information Technology

« Previous: 2. Creative Practices
Suggested Citation:"3. Advancing Creative Practices through Information Technology." National Research Council. 2003. Beyond Productivity: Information Technology, Innovation, and Creativity. Washington, DC: The National Academies Press. doi: 10.17226/10671.
×

3
Advancing Creative Practices Through Information Technology

How can computer scientists support new artistic and design practices? How is computer science (CS) research and development being stimulated and altered by emerging practices at the intersection of information technology (IT) and the arts and design? What are the prospects for new research directions that are interesting and useful for both computer science and the arts and design? Answering these questions requires a close look at the relationship between IT and the arts and design. That examination begins with this chapter, which focuses on the design, applications, and implications of the tools of IT from the perspective of the arts and design worlds, and it continues in Chapter 4.

STRANGE BEDFELLOWS?

One of the more obvious ways in which IT and the arts and design interact is in the use of technology to extend the expressive range of and modes of access to existing genres of the arts and design: Examples include Web-based art and hypertext, opera staging using new sensing and video technologies, musical compositions that feature both newly created instruments and interaction styles, and textile design and production based on digital weaving techniques. Given experimentation to date, it is clear that new tools developed by computer scientists can be immediately applied by artists and other creative practitioners within a wide array of contexts.

But there are further important implications of information technology and creative practices (ITCP) for computer science research and development. Box 3.1 provides context on the nature of that research for those who may be unfamiliar with it. Rather than using computational technology as black boxes for arts and design applica

Suggested Citation:"3. Advancing Creative Practices through Information Technology." National Research Council. 2003. Beyond Productivity: Information Technology, Innovation, and Creativity. Washington, DC: The National Academies Press. doi: 10.17226/10671.
×

BOX 3.1 Computing, Computer Science, and Research

Computing is rooted in the discipline of computer science,1 which studies information and computational processes—including the representation, implementation, manipulation, and communication of information.2 There are relatively few inherent natural limitations in computer science, as compared with other science and engineering fields such as physics, chemistry, and mechanical engineering. This means that the definitions—and, by extension, the capabilities—of computing and IT often do not have an obvious finite upper bound for expansion. There are, however, practical constraints on the available capabilities of computer hardware and software. In addition, the growing reach and complexity of computers and networks (such as the Internet) have heightened the risk and impact of system failures and created formidable challenges in areas such as security and the management of intellectual property. Efforts to improve computing capabilities are central to IT research. The key intellectual themes in computer science and engineering are algorithmic thinking (i.e., about rules for processing information), representation of information, and computer programs.3 Some IT research lays out principles or constraints that apply to all computing and communications systems; other studies focus on specific IT systems, such as user interfaces.

1  

The history of computing (which involves other fields such as electrical engineering) has been widely documented; see, for example, Computer Science and Telecommunications Board, National Research Council, 1999, Funding a Revolution: Government Support for Computing Research, National Academy Press, Washington, D.C.; and the public television documentary Triumph of the Nerds, transcript available at <http://www.pbs.org/nerds>.

2  

These aspects of computer science will be discussed in the forthcoming report of the Committee on the Fundamentals of Computer Science—Challenges and Opportunities, Computer Science and Telecommunications Board, National Research Council.

3  

See Computer Science and Telecommunications Board, National Research Council, 1992, Computing the Future: A Broader Agenda for Computer Science and Engineering, Juris Hartmanis and Herbert Lin, eds., National Academy Press, Washington, D.C.

tions, some are engaging in IT research as a form of art and design practice itself. This activity is less like historical arts research and more like computer science research, although it asks radically different kinds of questions and introduces a variety of methodologies generally unfamiliar to computer scientists. In certain areas of research, particularly human-computer interaction and artificial intelligence, there may be a convergence developing between new trends within CS research and the work of “outsiders” who bring in fresh perspectives. Unlike the use of computers for particular applications, this intersection of art and design and IT research leads to some deep and fundamental rethinking of CS research and what it is about in the first place. The intended outcomes go beyond making new tools for art and design practice, though that may be one outcome, to arrive at a fundamentally new way to do research—a true hybrid.

Both of these perspectives on interaction are important for the future of ITCP. In practice, they are intermingled. On the one hand, developing tools for new kinds of practices can lead to fundamental insights into the tool-development process. On the other, artists and designers who get their hands dirty in fundamental CS research are

Suggested Citation:"3. Advancing Creative Practices through Information Technology." National Research Council. 2003. Beyond Productivity: Information Technology, Innovation, and Creativity. Washington, DC: The National Academies Press. doi: 10.17226/10671.
×

able to build new tools and applications that can be useful for the arts and other creative domains.

The implications of this new medium for artistic and design practice run deeper than simple application. In previous communications revolutions, a new medium that was widely adopted not only added new possibilities for artistic and design expression, but also changed the way older media were used.1 Communication media influence the relationship between sense and bodily skill, and they alter the way in which artists and designers reason or feel about time and space.

If one were to think of such a shift as a radical break, in which a developing medium brings an entirely new art form into existence with little relationship to historical precursors, the naïve response for computer science researchers would be simply to generate as many new-media forms as possible as a way of advancing the opportunities for art and design practice. One technically oriented current in the contemporary avant-garde has indeed sought to push technology forward, in order to discover new expressive possibilities and genres that can be conceived only with the help of advanced technologies. Technological advances exert a strong pull, challenging artists and designers to conceive new expressive forms that can take full advantage of ever-increasing processing speeds and bandwidth rates. Both this push on and pull of new technologies focus on the new possibilities created by those technologies, rather than on the needs and perspectives of art and design practices using “old” (and by implication out-of-date) media.

It is a mistake to overemphasize the entirely new digital worlds that are uniquely possible using computers, as though the adaptation of already existing content or art forms to the new medium were only a lesser, transitional stage on the way toward the more significant discovery of purportedly new, essentially digital art forms.2 What tends to be overlooked, both by the modernist artist’s and designer’s technology push and the information technologist’s pull for advanced content, are the subtle and by no means trivial processes of change occurring in the traditional art and design forms as they adjust to and begin to find their own responses to information technologies. See Box 3.2 for an overview of how technology has influenced music while traditional activities have persisted.

Precisely because these developments are less visible, whether judged by criteria of radical artistic and design novelty or technical

1  

The rise of broadband continues that phenomenon. For example, one recent study found that, because of Internet use, 37 percent of broadband users watched less television, 31 percent spent less time shopping, and 18 percent spent less time reading newspapers; nearly 90 percent said that the Internet had improved their ability to learn. See <http://www.pewinternet.org/reports/toc.asp?Report=63>.

2  

See J.D. Bolter and R. Grusin, 1999, Remediation, MIT Press, Cambridge, Mass., pp. 48-50. Note that this situation may differentiate the arts from various practical activities (e.g., office and factory work), where early mechanization was, indeed, a step toward broad reconceptualization of different activities. Tradition and continuity are recognized across the arts.

The naïve response for computer science researchers would be to generate as many new-media forms as possible for art and design practice.

Suggested Citation:"3. Advancing Creative Practices through Information Technology." National Research Council. 2003. Beyond Productivity: Information Technology, Innovation, and Creativity. Washington, DC: The National Academies Press. doi: 10.17226/10671.
×

BOX 3.2 Technology, Music, and the Evolution of Expectations

Consider the history of music. Early music was not written—there was only the performance. The evolution of the musical score, a standardized interface between the composer and the performer, made it possible for the composer to achieve wider visibility by having his music performed numerous times by different groups. One tradeoff for this wider distribution was limited scope—only music that corresponded to this standard score could be written down (excluding all but Western music). Another was that the performance was still transient. It is said that Bach and Mozart were wonderful performers, but records are preserved mainly in listeners’ accounts, and in the occasional attempts by both masters to notate transcripts of their more spontaneous fantasy.

With the advent of recording, the performance itself could be captured. And the relationship between the composer and performer shifted. More improvisational styles such as jazz, blues, and rock can now be captured as a performance, not as a score, and the performer of current popular music is expected to perform his own music, not “cover” the music of others. We are back to the era of Bach or Mozart, but with global scale—the benefits of wider distribution now extend to other forms of music that cannot be written down. In turn, recording technology also enabled a new form of music—music that was created in whole or in part with synthesized, recorded, electronic signals, defying both transcription and performance. Thus, the introduction of recording raised new questions about music as a creative process and the definition of a live performance. There is something incongruous about attempting to trigger all the pleasure and excitement of a live gathering of an audience to look at a computer and a bunch of speakers up on stage. To some extent, of course, this is because we know that there will be no spontaneity in the performance— that creativity has already happened. This is what recording did to the creative aspect of music.

The computer is now doing much more. First, the computer can now be the performer. Digital technology allows composers to craft their own instruments and to create a performance using only the computer. Computer-driven pianos now replay a performance with many of the nuances of the original pianist’s keystrokes. Interactive computer programs can participate in the performance, producing and manipulating sounds in response to a performer’s actions. Digital technology, through interactivity, helped bring spontaneity back into the performance of electronic music. What seems to be evolving is a remodularization of the creative process, the resolution of which is a creative act in its own right, and one that cannot escape the feedback and interaction with the listener. The role of the listener, the audience, illustrates how advances are gated by a social process.

Secondly, the computer can capture a performance and make it permanent. What the recording industry did for music, the computer can in principle do for a more dynamic and multimedia creation. Consider also the emergence of the disc jockey who does a live remix of pre-existing music (including electronic music): This seems to be a reassertion of the persistent value of the spontaneous process of performance, involving real-time feedback between the performer and the audience. Here, again, questions arise about how to view the “dilemma” of the computer as a part of the creative process.

progress, they receive less attention than more obviously glamorous showcases do. But these subtle developments are no less important for the long-term ecology of digital culture, suggesting limitations of, and different possibilities for, the development of technology as a medium.

For example, tacit knowledge—unformalized and probably unformalizable knowledge such as design methodologies or embodied skills such as drawing or dancing—has always played an impor

Suggested Citation:"3. Advancing Creative Practices through Information Technology." National Research Council. 2003. Beyond Productivity: Information Technology, Innovation, and Creativity. Washington, DC: The National Academies Press. doi: 10.17226/10671.
×

tant role in the arts. Yet IT has by its nature had severe difficulty codifying the most subtle and refined of such artistic practices. The ineffable human feel can be simulated acceptably through clever tricks, but the danger all too often is that consumers of technology, including artists and designers, will accommodate themselves to the reduced expressive bandwidth afforded in easy-to-use interfaces, as discussed below.3 The emerging research paradigm for embodied interaction in human-computer interaction (HCI) is one opportunity for a different style of interface perhaps more compatible with highly skilled art practice.4

TOOLS NEEDED TO SUPPORT CREATIVE WORK: HARDWARE AND SOFTWARE

Computer and communications hardware and software are the tools of ITCP and the means by which almost all digital media are created and manipulated. These tools can do many things,5 each and all of which may be embraced in ITCP. A simple list of gross capabilities would include:

  • Automation of processes such as drawing, composing, editing, and so on (assorted software);

  • Handling, representing, and displaying or performing information (databases, browsers, displays and speakers, printers, projection systems, and so on);

  • Analysis of information and phenomena (visualization and sonification, modeling and simulation, artificial intelligence and learning systems);

3  

For sociological analysis of creative effects of the commoditization of musical instruments in digital forms, see P. Théberge, 1997, Any Sound You Can Imagine: Making Music/Consuming Technology, University Press of New England, Hanover, New Hampshire.

4  

Embodied interaction has been a theme in digital arts at least since Myron Kreuger’s VideoPlace of the early 1970s. See P. Dourish, 2001, Where the Action Is: The Foundations of Embodied Interaction, MIT Press, Cambridge, Mass.; and P. Dourish, 1999, “Embodied Interaction: Exploring the Foundations of a New Approach to HCI” (see <http://www.ics.uci.edu/~jpd/publications>). An example of a new body-centered interface for art is described in Steven Schkolne, Michael Pruett, and Peter Schröder, 2001, “Surface Drawing: Creating Organic 3D Shapes with the Hand and Tangible Tools,” pp. 261-268 in Proceedings of CHI 2001, ACM Press, New York.

5  

Broad interpretations include not only components and artifacts but also their theoretical underpinnings and digital content. The Computing Research Repository, for example, lists 34 subject areas; see <http://xxx.lanl.gov/new/cs.html>. The Computing Research Repository is an online archive of computer science research results that uses the Internet to allow access to technical reports, conference papers, and other work on a near-real-time basis.

Suggested Citation:"3. Advancing Creative Practices through Information Technology." National Research Council. 2003. Beyond Productivity: Information Technology, Innovation, and Creativity. Washington, DC: The National Academies Press. doi: 10.17226/10671.
×

BOX 3.3 One Set of ITCP Technologies

One perspective on technologies relevant to ITCP work is provided by Carnegie Mellon University’s Entertainment Technology Center (ETC), which offers a master’s degree jointly conferred by the College of Fine Arts and the School of Computer Science. Entertainment technology “requires a fluid definition, necessitated in large part by advances in technology that are making possible ever-new entertainment experiences and venues. What was meant by the phrase entertainment technology as recently as a year ago requires redefinition in light of recent developments in both technology and entertainment.”1 Nevertheless, the following listing is provided:

  • Networked and free-standing interactive computer games

  • Avatar creation and utilization

  • Massive multiplayer online games

  • Digital entertainment

  • Specialty venues such as theme parks

  • Motion-based rides

  • Console and PC interactive game design

  • Creation of unique input devices

  • Virtual reality utilizing head-mounted displays

  • Other forms of virtual reality technology

  • Wearable computing for entertainment purposes

  • Massive immersive display environments

  • Interactive robot animatronics

  • Synthetic interview technology

  • Speech recognition

  • Augmented reality

  • Telepresence for entertainment and education purposes

  • Digital production and postproduction

  • Sound synthesis, surround sound, three-dimensional sound, and streaming audio

  • Development of haptic devices (e.g., force feedback)

  • Entertainment robotics.

  • Connection to the physical world (sensors, microphones, digital cameras, actuators, robots, human interfaces, systems for interactivity); and

  • Communications (telephony, television, the Internet, and assorted underlying capabilities, from wireless to broadband connectivity).

Ultimately, IT acts on information typically associated with products of the human mind (pictures, music, and ideas), although IT must also address data and information in less processed, intermediate states.6

6  

This concept is derived from Computer Science and Telecommunications Board, National Research Council, 1992, Computing the Future: A Broader Agenda for Computer Science and Engineering, Juris Hartmanis and Herbert Lin, eds., National Academy Press, Washington, D.C.

Suggested Citation:"3. Advancing Creative Practices through Information Technology." National Research Council. 2003. Beyond Productivity: Information Technology, Innovation, and Creativity. Washington, DC: The National Academies Press. doi: 10.17226/10671.
×

Human-computer interaction specialist Ben Shneiderman argues that IT for creativity support falls into eight categories: searching, visualizing, consulting, thinking, exploring, composing, reviewing, and disseminating.7 Because the currency of IT is bits, IT tools can handle any mode or medium, and they can integrate any combination of media— although sophistication has costs in complexity and dollars, and for technical and/or economic reasons, not all desired effects are possible (see “Economic Realities” below). For a concrete illustration of a tool set inspired by entertainment applications, see Box 3.3. Lists like these show the broad range of IT that may be linked to some aspect of the arts or design through ITCP; no list conveys all the creative possibilities inherent in the use of IT.

By definition, tools are supposed to be helpful, but like other tools, IT tools have shortcomings. The insights into the nature of IT that collectively constitute fluency, as described in Chapter 2, include an understanding of the limitations of the tools and the constrained nature of the typical software design. The pervasive hype about IT in the mass media may, by contrast, feed unrealistic expectations. Although artists and designers share frustrations with other user groups, their perspectives, like those of other users, may help illuminate new paths for IT research and development. This situation was recognized in a recent special issue of a leading computer science journal:

Many significant advances in research on human creativity have occurred, yet today’s tools often contain interface elements that stymie creative efforts. A discontinuity exists between technology tools and our ability to interact with them in natural, beneficial, and most importantly, for this discussion, creative ways.8

Tools vary in terms of the computing and programming skills required to use them. As long as the tools required to produce computer-mediated work are programming tools, the result will be programmer-created design. That is not a bad thing—and in some cases the result(s) can be wonderful. But it does mean that an investment must be made in learning, which is somewhat like the requirement to master other tools used by artists, but also different, because of factors such as the range of features and capabilities available from software and the relatively rapid change in technology. There is a great distance from the paintbrush or piano to programming in C++. As emphasized in Chapter 2, fluency can provide a middle ground between simple acceptance of a tool and expertise in programming.9 Although the

7  

See Ben Shneiderman, 2002, “Creativity Support Tools,” Communications of the ACM 45(10): 116-120.

8  

Winslow Burleson and Ted Selker, 2002, “Introduction (Special Issue: Creativity and Interface),” Communications of the ACM 45(10): 88-90.

9  

Of course, new approaches to programming that simplify it may be helpful— assuming that the results do not present the same concerns that software packages do about built-in constraints.

There is a great distance from the paintbrush or piano to programming in C++.

Suggested Citation:"3. Advancing Creative Practices through Information Technology." National Research Council. 2003. Beyond Productivity: Information Technology, Innovation, and Creativity. Washington, DC: The National Academies Press. doi: 10.17226/10671.
×

discussion in this chapter focuses on software and to a lesser extent hardware for creating artifacts and performances, it should be noted that other tools consist of content (images, sounds, text, and so on) repositories. There the concerns center on the accessibility of the content—involving indexing, permission to use (or ease of obtaining same), and so on. Yet still other tools support public access.10

HARDWARE AND SOFTWARE TOOLS: A MIXED BLESSING

Information technology has obviously proved useful and accessible enough to give rise to ITCP in many guises. In the process, a number of observations have emerged about the nature of IT and the adaptations that artists and designers make in using it. The concerns gleaned from artists and designers help to explain why it takes a long time to integrate a new technology into the making of non-trivial art or design work.

Because all computers are universal machines, more advanced designs can replace simpler ones, providing improved performance without changing the basic functionality of computation. Software is even more mutable in that it has almost no physical constraints. Because of rapid changes in hardware capabilities, software rarely matures to a stable configuration. On the positive side, this pace of change provides frequent opportunities to incorporate improved support for creativity in new systems. But for most users, especially in relatively resource-poor areas such as the arts and humanities, this constantly changing tool set is difficult to master. For well over a decade, it has been the lament of artists and designers that the intellectual and financial demands of constantly updating tools and playing technological catch-up results in low-quality work and burnout.11 Smart artists, observed a reviewer, resign from the Moore’s law rat race. These conditions suggest that it is reasonable to expect a wide range of willingness and ability among artists and designers to retrain and to upgrade their tools; how this will affect ITCP remains to be seen.

Developers of software tools that can support creative practices have a number of variables to consider, all of which may affect the

10  

See tool characteristics in Sharon L. Greene, 2002, “Characteristics of Applications That Support Creativity,” Communications of the ACM 45(10): 100-104.

11  

An anecdote shared by a reviewer featured a senior colleague who reported that when he discovered the Amiga computer, he felt sure he had found the tool with which he would make his magnum opus. After several years of learning about and constantly upgrading his Amiga tools, he had become an Amiga expert but as yet had produced no work. Much to his chagrin, the machine then became obsolete. More generally, in the mid-1990s a committee of media arts faculty prepared a document, colloquially referred to as the “‘burnout” document, that outlined the new and unrecognized loads on media arts faculty. It was endorsed by the College Art Association, Inter-Society for the Electronic Arts, and Special Interest Group on Computer Graphics and Interactive Techniques.

It takes a long time to integrate a new technology into the making of non-trivial art or design work.

Suggested Citation:"3. Advancing Creative Practices through Information Technology." National Research Council. 2003. Beyond Productivity: Information Technology, Innovation, and Creativity. Washington, DC: The National Academies Press. doi: 10.17226/10671.
×

ways in which users interact with the tools—and through the tools, their own work—and the ease with which users can produce original or even groundbreaking works with those tools. To name just a few, tool developers dictate the user conceptual model (or metaphors) exposed by their tools, the amount of structure supported or imposed on the work process and product, the number and kinds of different presentations and representations of content supported, the kinds of manipulations directly implemented, the openness and extensibility of the tool at various levels, and the levels of abstraction afforded. The developers’ decisions often reflect attempts to make the system easy to use,12 or simplifying assumptions about what users want, but they can also greatly affect what users produce.13 Very little research has explored the relationships between these design decisions and the fitness of the tool for various kinds of ITCP work—what follows are some observations made by the committee.

In an effort to get work done, it is only natural to follow the path of least resistance established by the tools that are available, so these tools play a very important part in how users conceptualize their work and assess their options. The tools may also leave traces: Architects may look at a building and detect which design tools were used, and musicians may hear new pieces and detect which composition tools were used.14 Thus, IT may act as a flywheel in at least some contexts of its use. Artists with IT fluency recognize these practical realities and their roots in the values and procedures of computer science; they work with or around the limitations of the tools.

In the popular style known as object-oriented programming, a programmer creates a new kind of object (concretely, a data structure that describes something) and defines the operations that other programmers can perform on it. Other programmers that use the object cannot get at the data structure itself, but only at the operations that the object-creator defined. This gives the object-creator the ability to come back later and change the actual data structure (perhaps to make it more efficient) without having to coordinate with any other pro

12  

An early illustration in a number of (non-artistic) fields was statistics; statistical software (and later spreadsheets) popularized and helped to disseminate a variety of quantitative methods used in different applications. More recently, software packages have begun to implement neural networks, face recognition, data acquisition, or image display. Because these functions can be technically difficult to create, careful packaging can allow others to use advanced technology with less effort.

13  

For example, the literary theorist has written, with reference to hypertext authoring environments: “The strength of metaprograms is that they take away most of the pain involved in programming an application from scratch; . . . [their] weakness is that they limit the programmer by presenting a predefined range of operations that the programmer must use. . . . This may be compared to pre-modern modes of authorship, in which the author could use predefined paradigms to produce a genre text, without much creative effort.” See Espen J. Aarseth, 1997, Cybertext: Perspectives on Ergodic Literature, Johns Hopkins University Press, Baltimore, Maryland, p. 173.

14  

In other contexts, such as education, people have noted how the presentation software PowerPoint shapes and constrains their choice of content and organization for public speaking.

Tool developers dictate the user conceptual model (or metaphors) exposed by their tools.

Suggested Citation:"3. Advancing Creative Practices through Information Technology." National Research Council. 2003. Beyond Productivity: Information Technology, Innovation, and Creativity. Washington, DC: The National Academies Press. doi: 10.17226/10671.
×

It is much easier to design a tool for a well-defined task than for non-deterministic, creative work.

grammers. The drawback is that the user can perform only the operations the object-creator thought of. If the user wants to add a new operation, he or she cannot, if doing so requires getting at the base data representation. It is much easier to design a tool for a well-defined task than for non-deterministic, creative work, which cannot be reduced to a set of tasks.

As one example of how these design decisions can affect the utility of tools, consider an image-editing program that provides a “blur” feature with a capability for “more” and “less” control. Technically, this is a simple mathematical convolution on the image, but the tool designers chose not to expose the numeric parameters for this operation. The benefit is that an operation assumed to be common—blurring an image—is simplified. The tradeoff is that other convolutions— which might prove interesting or useful to a user—are impossible without some other software mechanism.

One design point that provides good support for explorations of ITCP is to use a small number of concepts in a general and flexible way combined with openness and interoperability with other tools. A good example is the interactive music language “Max,” a full visual programming environment that has developed a community with at least three ranks of creative users. The least technical of these would be a performer who develops skills as an interactor with a particular set of patches (configurations), much like a musical performer would practice her instrument. Users may then learn to make their own programs, by patching together already existing objects in unique ways. Such patches then become instruments that permit a high degree of virtuosity for use within particular genres (like techno-music or interactive dance).

Creating a good tool requires an understanding of the problem area and often experience at the cutting edges of artistic and design and technical disciplines, just to understand the underlying concepts and representations that the tool will address. Tool making also can benefit from combining technical expertise in designing and implementing software with a healthy dose of common sense to arrive at something general enough to be useful but simple enough to be used.15 Problems sometimes arise when tool design assumes too much separation of idea from expression.16

15  

People familiar with computer science might consider Donald Knuth as an early explorer of ITCP. Venerated for his understanding of software, Knuth has done work on fonts and electronic composition that is recognized broadly.

16  

Wright points out that the separation between idea and expression in software— which can degenerate to the software supplying all the ideas—is one example of a more general problem with creative software tools. These tools most commonly “aim to mirror internal creative process by organizing it into an external data process or structure. The software is a system of menu commands and options which seeks [is designed] to match an internal model of creativity as a process of decision making that seeks to approximate an ideal artistic goal.” But artists and designers “do not know in advance what they want before they start—the creative process is actually a process of

Suggested Citation:"3. Advancing Creative Practices through Information Technology." National Research Council. 2003. Beyond Productivity: Information Technology, Innovation, and Creativity. Washington, DC: The National Academies Press. doi: 10.17226/10671.
×

A tool may offer just the right functionality, but not at the right component level. For example, an image can be manipulated through the use of a comprehensive interactive application such as PhotoShop, or by typing a command to run a specific program such as one of the Unix ppm utilities,17 or by calling a function in an image processing library. A software library is of little use to a non-programmer, whereas a full-blown application is of little use in creating a custom software program. Sometimes, a full-blown application can be made to perform like a custom software program through scripting, and sometimes software libraries or other components can be integrated with an application via a plug-in capability (see below). More could be accomplished with tools designed to suit a range of different tasks. Ensuring this flexibility calls for either a variety of tools or a hierarchical structure that allows access to tools at various levels of abstraction, permitting a migration path from intelligent enhancement at the novice level to customization for experts (as illustrated by the language “Max” described above).

Because creativity is associated with novelty, comprehensive tools for creative work will be neither possible nor necessary to develop, any more than it is necessary for a pencil to include all functions for drawing. It will always be the task of the innovator to create new tools from components, to create new applications, and to create new artifacts by using tools in unanticipated ways. In this respect, ITCP will draw from both artistic and design and computer science traditions with experimentation. Small professional communities that share experiences and talent often develop effective tools—this phenomenon is evident in the growth of computational science and the development of the Web by and for physicists, and it has been observed in humanities fields as well.18 Many small tools are developed by individuals or small teams that are able to keep their tools focused and coherent.

Design choices related to tool extensibility may be particularly important for broadening participation in ITCP and tool development. Programming language extensions (also called scripting languages) allow end users to customize applications. For example, computer music and animation programs often use text-based scripts or scores to describe music and images. Users can create scripts either by editing text directly or by writing programs to generate scripts. Another way to customize tools is with a plug-in, which is a software compo

   

playing and ‘visual thinking’ that leads to a variety of interim ‘solutions’ and modifications of the original ‘problem’ . . . .” See Richard Wright, 1999, “Programming with a Paintbrush: A Study in the Production Culture of the Moving Image,” Filmwaves, Issue 12.

17  

These programs perform a wide range of image-processing tasks including format conversion, scaling, filtering, and color adjustment.

18  

See American Council of Learned Societies, 1998, Computing and the Humanities: Summary of a Roundtable Meeting, Occasional Paper No. 41, available online at <http://www.acls.org/op41-toc.htm>.

Comprehensive tools for creative work will be neither possible nor necessary.

Suggested Citation:"3. Advancing Creative Practices through Information Technology." National Research Council. 2003. Beyond Productivity: Information Technology, Innovation, and Creativity. Washington, DC: The National Academies Press. doi: 10.17226/10671.
×

nent that extends an application by implementing a new function—a much simpler approach than writing an entirely new application. Plug-ins are common for image- and audio-processing programs; they exist for such popular design tools as PhotoShop and Auto-CAD.19 Although commercial image and video editors may not support functions desired by researchers, artists, and designers working at the creative frontiers, a plug-in architecture enables users to extend editors with new custom functions. Note that this capability presents its own challenges to implement: A program like PhotoShop that allows other programmers to create plug-ins has to reveal a lot of its internal structure. This allows more third-party creativity (by those who have the programming skills), but it constrains the maintainers of PhotoShop to preserve all this internal structure as they move from revision to revision. Balancing these tensions is an act of creativity in applied computer science. Generalizing from the discussion of plug-ins, creativity can be supported by systems that are extensible, a feature that is often associated with a modular design (a plug-in is a module), to support a broad range of users and their tasks. Modular designs require software “hooks” to which new capabilities can be added. Recently, however, as programs have grown to help users accomplish more tasks with less expertise, they have tended to hide their inner workings and to inhibit flexibility to do things that were unanticipated by the designers of the program.

Open application program interfaces, scripting languages, and plug-in architectures are no panacea, however. Decisions about what to expose (the level of abstraction, the set of functions, access to data, and so on) will still influence the ways in which users can extend a tool. Also, taking advantage of these forms of extensibility requires expertise in programming. Research could illuminate alternative approaches, exploring how to make extensibility and alternative configurations more cost-effective and how to implement virtual tuning knobs that allow artists and other users to make various adjustments.

In the hardware area, sensors20 often require significant engineering expertise and are difficult to interface with popular operating systems. Ready-to-use designs, circuit boards, and specialized operating systems can simplify the use of hardware sensors in many interesting applications,21 and there is evidence that at least elements of a

19  

An example is the RealOne Player (by RealNetworks) for downloading and viewing video segments on the Internet. See <http://www.real.com>. Other examples include plug-ins that allow browsers to read proprietary files, such as those in Portable Document Format (PDF).

20  

Sensors can be applied in many contexts relevant to ITCP work, as described in subsequent sections of this chapter.

21  

See, for example, information on prefabricated robot microcontrollers, such as the Handyboard, at <http://www.handyboard.com/>.

Suggested Citation:"3. Advancing Creative Practices through Information Technology." National Research Council. 2003. Beyond Productivity: Information Technology, Innovation, and Creativity. Washington, DC: The National Academies Press. doi: 10.17226/10671.
×

relevant tool base are emerging in IT research circles that may be valuable for ITCP.22

Inasmuch as artists’ needs depart from commercial offerings, or inasmuch as specialized tools are kept proprietary, as with some instances in the competitive animation arena, artists’ work in ITCP may require support for experimentation in tool development. New software tools could be developed that would enable users to build their own software tools—collaboration between artists and designers and computer scientists could aim at a meta-toolkit that would offer ease of use plus flexible, extensible results, and an Internet-accessible repository of available software and hardware tools as well as guidance on what it takes to use them would broaden access and experimentation.23 Support for programming by people lacking full-fledged programming skills could be part of this kit.24 For effective development to take place, it is important to understand what shape the desired ITCP software will take—what interfaces it should provide, what functions it should support, and so on. Hence, building research prototypes would be a reasonable first step; such prototypes are going to be built by motivated practitioners or researchers rather than by commercial software developers—who may, of course, transform attractive prototypes into products.

As with other instances of user-generated IT tools or the broader phenomenon of popular culture finding new purposes for tools,25 ITCP tool-development efforts may prove to have broader appeal. The rationale for encouraging this activity echoes the rationale for encouraging research on usability: Steps designed to benefit a small constituency (e.g., people with various disabilities) may prove to benefit users in general.26 Research oriented to enhancing creativity is high-leverage.

22  

See, for example, the TinyOS work based at the University of California at Berkeley. It aims at developing an open experimental software/hardware platform for network embedded systems technology research that will dramatically accelerate the development of algorithms and services and their composition into challenging applications. Small, networked sensor/effector nodes are developed to ground algorithmic work in the reality of operating with numerous, highly constrained devices. See <http://webs.cs.berkeley.edu/tos/>.

23  

Rhizome.org, for example, has incubated an open-source software archive for artists (artists can submit open-source code to an online database, download code, discuss it, and so on).

24  

There may be capabilities to borrow from the education community, such as Logo and its derivatives, or from efforts aimed at broadening use of the Web. Or there may be insights to glean from other kinds of tools. For example, Mathematica is a program to perform mathematics. An important component of Mathematica is its programming language, which allows tedious tasks to be automated, presumably freeing up some of the user’s time for more exotic pursuits. See <http://www.mathematica.com>.

25  

As William Gibson puts it in Neuromancer (Ace Books, New York, 1984), the street finds its own uses for things.

26  

Computer Science and Telecommunications Board, National Research Council, 1997, More Than Screen Deep: Toward Every-Citizen Interfaces to the Nation’s Information Infrastructure, National Academy Press, Washington, D.C.

Collaboration between artists and designers and computer scientists could aim at a meta-toolkit that would offer ease of use plus flexible, extensible results.

Suggested Citation:"3. Advancing Creative Practices through Information Technology." National Research Council. 2003. Beyond Productivity: Information Technology, Innovation, and Creativity. Washington, DC: The National Academies Press. doi: 10.17226/10671.
×

BOX 3.4 Sketchy Interfaces

“Sketchy” interfaces provide the ability to describe objects with ambiguous types, sizes, shapes, and positions quickly. The ambiguity encourages a focus on larger concepts, rather than details such as font size or precise alignment. James Landay has developed a tool called “Sketching Interfaces Like Krazy” (SILK) that allows designers to create prototypes of interface designs by sketching. The difficulty with previous interface tools is that they have required designers to specify color, shape, size, and orientation of the elements of an interface precisely from the start. This requirement focuses the designer’s and his or her test users’ attention on aspects of the design that are irrelevant in the early stages of design. As a consequence, many designers prefer old-fashioned paper sketches. Based on a survey of interface design practices, SILK attempts to combine the positive imprecision of paper sketches with the benefits of storage, searchability, and interactivity that information technology can provide. Designers can roughly sketch and storyboard their ideas on a sketch pad, while SILK automatically makes the sketches interactive, allowing them to be used for testing.1

1  

For further discussion on SILK, see <http://www.cs.berkeley.edu/~landay/research/publications/CHI96/video.html> or James A. Landay, 1996, Interactive Sketching for the Early Stages of User Interface Design, Ph.D. Thesis, Carnegie Mellon University, Computer Science Department, available online at <http://reports-archive.adm.cs.cmu.edu/anon/1996/abstracts/96-201.html>. Also see <http://www.computer.org/computer/homepage/march/cov_feat/0301silk_side.htm>; James A. Landay and Brad A. Myers, 2001, “Sketching Interfaces: Toward More Human Interface Design,” IEEE Computer 34(3): 56-64; and James A. Landay and Brad A. Myers, 1995, “Interactive Sketching for the Early Stages of User Interface Design,” pp. 43-50 in Proceedings of CHI ‘95, Denver, Colo., ACM Press, New York.

SUPPORT FOR FLEXIBILITY, EXPERIMENTATION, AND PLAY

IT facilitates the rapid design and creation of digital artifacts and their equally fast refinement.27 Creative design is often an iterative process in which many ideas are considered. Sometimes a sketchpad is much more effective than a computer-aided design (CAD) tool because the precision and detailed specification required for CAD are not needed in the early stages of design. The computing literature gives a number of examples of “sketchy interfaces” that allow designers to make rough prototypes to explore a design space rapidly. There is also evidence that “sketchy” designs tend to elicit more higher-level comments (“Why do you need this function?” as opposed to “I don’t like the color scheme”). These and other examples illustrate how tool designers vary the kinds of structures imposed on the work process and product in their tools, often to facilitate experimentation, improvisation, and flexibility. See Box 3.4.

27  

“Support pain-free exploration and experimentation (a ‘sandbox’ mode). There should be an easy way to undo and redo all or part of one’s work. There should be no big penalties for mistakes, and there should be meaningful rewards for success. There should be immediate and useful feedback for one’s actions, promoting a sense of control.” See Ben Shneiderman, 2002, “Creativity Support Tools,” Communications of the ACM 45(10): 102.

Suggested Citation:"3. Advancing Creative Practices through Information Technology." National Research Council. 2003. Beyond Productivity: Information Technology, Innovation, and Creativity. Washington, DC: The National Academies Press. doi: 10.17226/10671.
×

In the right hands, the right computer tools can enable improvisation, which is an important expression of creativity in response to a dynamic situation. Just as a jazz musician improvises by listening and responding to a dynamic musical context, all artists improvise with media in creative ways; on the computer science side, the synthetic nature of software implies that improvisation played a role in its development. At least for some artists and designers—and for many computer scientists and other scientists and engineers—computers carry out computations to explore “what if” scenarios. Such computer-based experimentation inspires new ideas, driving the interplay among design, simulation, surprise, and creation. Tools support improvisation when they offer interactive design, revision, and elaboration of partial specifications. Doing this successfully requires careful consideration of the tradeoffs between imposed structure and clean-sheet approaches, among other design parameters. A highly structured musical tool with a rich model of tonal music may be excellent for certain kinds of improvisation and experimentation, but not for others, for example. See Box 3.5.

THE INTERNET AND THE WEB

The Internet28 is particularly useful in ITCP work because of several unique features that set it apart from traditional communications systems, such as the public switched telephone network and the cable and broadcast television systems.29 The Internet’s design encourages innovation at the edges by users, allowing a relatively unrestricted set of applications to run over it. By contrast, traditional networks are centrally developed and managed and historically have limited what users can do with them. Connection, interconnection, and innovation in facilities and services are relatively easy with the Internet, making it possible to use the underlying communications infrastructure more efficiently and inexpensively. These factors have generated a pattern of innovation in Internet technologies and uses associated with a culture of cumulative knowledge-building. Thus, not only are personal computers and larger computer systems attached to the Internet, but so also are televisions, telephones, personal digital assistants, and

28  

The Internet has both computing and communications components. The networks that constitute the Internet are composed of communications links, which carry data from one point to another, and routers, which direct the flow of communications between links and, ultimately, from senders to receivers. Routers are computer devices located throughout the Internet that transfer information from a source to a destination. And of course, users access the Internet from a computing device, which may be in a form other than a conventional computer.

29  

For the history of the Internet, see the home page of the Internet Society at <http://www.isoc.org>. See also Katie Hafner and Matthew Lyon, 1996, Where Wizards Stay Up Late, Simon and Schuster, New York; and Computer Science and Telecommunications Board, National Research Council, 1999, Funding a Revolution: Government Support for Computing Research, National Academy Press, Washington, D.C.

Suggested Citation:"3. Advancing Creative Practices through Information Technology." National Research Council. 2003. Beyond Productivity: Information Technology, Innovation, and Creativity. Washington, DC: The National Academies Press. doi: 10.17226/10671.
×

BOX 3.5 Improvisatory Interactivity in Music

In his briefing to the committee, George Lewis described his work with improvisational systems.1 Improvisation characterizes much of human activity, but it has not been explored nearly as much as related activities such as engineering, design, and even creativity generally. At its most basic level, improvisation is a series of reactions to situations, often intuitive, expressive, and spontaneous. Improvisation is about finding structure, not imposing it. In spite of the informal and unplanned nature of improvisation, it can be conducted with consummate artistry, and great improvisers develop their skills over a lifetime of work. Improvisation is an interesting metaphor for human-computer interaction.

Current technology, as practiced in computer music systems, has a long way to go before it will be considered highly skilled, intelligent, or creative. In fact, it can be argued that present systems are successful because of the skill of human performers. Humans can form plans at many time scales, from milliseconds to minutes, and humans are uniquely equipped to listen to and evaluate music as it is created. In contrast, machine musicianship is generally weak and at best skillful in a very narrow range of situations. However, there are interesting and humanly impossible tasks that computers can perform, creating some very interesting musical possibilities.

What does it mean, technologically, to build an improvisation system? First, any interactive system must have sensors and analysis functions. Second, the system must have some method of making high-level musical decisions and generating music. Third, the system must have performance skills, that is, the ability to transform musical information into real-time commands controlling pitch, amplitude, vibrato, and other expressive parameters of sound. Finally, the system must generate sound from this control information, also in real time.

Sensing can be accomplished using a microphone to “listen” to the human performer. Signal processing can detect pitch, amplitude, brightness, and other features, although this is currently restricted to monophonic instruments, those that produce only one tone at any given time. Sensing can also use non-acoustic means, such as optical or electrical position sensors in electronic keyboards, accelerometers, optical sensors, and even a standard computer keyboard and mouse. Once low-level sensor data is acquired, the system must interpret the data in musical terms.2 The system might parse the data into phrases, build histograms of pitch (indicators of keys and modulation), estimate tempo,3 or use statistical classifiers to detect emotion,4 style,5 or other musical information.

Once processed into some abstract representation of music, sensor data become an input to the improvisation process. The system performs decision making and planning at this stage based on input from sensors, memory of past events, and built-in knowledge.6 For example, George Lewis’s Voyager program is capable of generating music without any input or interaction, but when it senses certain trends in the human improvisation, it can modify its own performance to either go along with, or contrast with, the human musician. This aspect of the improvisation system often draws on formal theories of music and computer science. For example, the system might use Markov models to generate chord progressions, or 12-tone serial techniques to generate melodies. Rule systems are often used to react to sensor data, generating decisions and plans that in turn modify the music generation process.

As high-level representations of music are generated, the music must be performed in real time. Performance can include such things as voicing and orchestration, that is, deciding which synthetic instruments will perform which notes of the music, expressive modulation of various control parameters, and at the lowest level, scheduling accurately timed parameter updates to the music synthesizer. Some recent advances in the understanding of emotion in music, for example, allow a stream of music events to be transformed in ways that express anger, calm, happiness, or sadness.7

The final stage is the actual generation of sound. Early improvisation systems often relied on off-the-shelf synthesizers to generate sound from low-bandwidth musical instrument digital interface (MIDI)8 control information. Now that laptop computers can generate rich sounds in real time, many computer musicians have been attracted to the possibilities of direct control over sound at the signal-processing level. Systems often transform acoustic input from the human performer, giving the improviser a much more intimate and direct connection to the generated sound. In this case, the improvisation might focus more on

Suggested Citation:"3. Advancing Creative Practices through Information Technology." National Research Council. 2003. Beyond Productivity: Information Technology, Innovation, and Creativity. Washington, DC: The National Academies Press. doi: 10.17226/10671.
×

FIGURE 3.5.1 Andrew Schloss improvises with an interactive computer music system, using a radio drum and foot pedals. Image courtesy of the Banff Centre.

the character of the sound (timbre) rather than on abstract music structures of pitch and rhythm (Figure 3.5.1).

Many interesting problems arise in interactive improvisation systems. How do improvisers build on local, almost reflexive decisions to create masterful form at a more global level? Can this process be modeled in the computer to create a musical companion?9 Human perception of music includes sophisticated recognition of patterns and motives. Can systems be extended with better models of perception? Improvisers study and learn. Can computer systems also learn to improvise? Whether discussing biological or silicon systems, computation must take place at many time scales. A jazz drummer produces events with very high timing resolution relative to the speed of high-level cognitive processing. Similarly, computer systems must simultaneously deliver low-latency, hard, real-time signal processing while simultaneously performing high-level decisions at a more relaxed time scale.10

Current systems have not solved all of these problems, and they are generally weak at forming plans, performing high-level perception and recognition tasks, and offering sophisticated composition skills. Nevertheless, interactive computer music systems incorporating improvisation have established a new genre of contemporary music. These systems offer a model for human-computer interaction driven by shared goals, high-level task-oriented communication, and creative, situated decision making by both human and computer.

1  

See George Lewis, 2000, “Too Many Notes: Computers, Complexity and Culture in Voyager,” Leonardo Music Journal 10; George Lewis, 1999, “Interacting with Latter-day Musical Automata,” Contemporary Music Review 18 (Part 3): 99-112 (with accompanying CD); Kristin Palm, 2001, “Making a Point” (review of interdisciplinary, multiartist project at the Point Loma Wastewater Treatment Plant in San Diego, including a site-specific interactive videosonic installation by George Lewis), Metropolis (August/September); Guy Garnett, 2001, “The Aesthetics of Interactive Computer Music,” Computer Music Journal 25(1); Todd Winkler, 1998, Composing Interactive Music: Techniques and Ideas Using Max, MIT Press, Cambridge, Mass.; Ben Ratliff, 1997, “Improvisers Meet the Machines” (Review of “Voyager,” CD recording featuring the computer music of George Lewis), New York Times, October 14; and Joel Chadabe, 1997, Electric Sound: The Past and Promise of Electronic Music, Prentice-Hall, Upper Saddle River, N.J.

2  

See Robert Rowe, 2001, Machine Musicianship, MIT Press, Cambridge, Mass.

3  

See Masataka Goto, 2001, “An Audio-based Real-time Beat Tracking System for Music with or without Drum-sounds,” Journal of New Music Research 30(2): 159-171.

4  

See A. Friberg, E. Schoonderwaldt, P.N. Juslin, and R. Bresin, 2002, “Automatic Real-time Extraction of Musical Expression,” pp. 365-367 in Proceedings of the International Computer Music Conference—ICMC 2002, International Computer Music Association, San Francisco, Calif.

5  

See Roger B. Dannenberg, Belinda Thom, and D. Watson, 1997, “A Machine Learning Approach to Style Recognition,” in International Computer Music Conference, International Computer Music Association, available online at <http://www.cs.cmu.edu/~rbd/bib-styleclass.html#icmc97>.

6  

See Todd Winkler, 2001, Composing Interactive Music, MIT Press, Cambridge, Mass.

7  

See R. Bresin and A. Friberg, 2000, “Emotional Coloring of Computer-controlled Music Performances,” Computer Music Journal 24(4): 44-63.

8  

See Joseph Rothstein, 1995, MIDI: A Comprehensive Introduction, 2nd Ed., A-R Editions, Madison, Wisc.

9  

See Belinda Thom, 2001, A Customized, Interactive Melodic Improvisation Companion, Ph.D. Thesis, Carnegie Mellon University, Computer Science Department, Technical Report CMU-CS-01-138, Pittsburgh, Pa.

10  

See Roger B. Dannenberg and Patrick van de Lageweg, 2001, “A System Supporting Flexible Distributed Real-Time Music Processing,” pp. 267-270 in Proceedings of the 2001 International Computer Music Conference, International Computer Music Association, San Francisco, Calif.

Suggested Citation:"3. Advancing Creative Practices through Information Technology." National Research Council. 2003. Beyond Productivity: Information Technology, Innovation, and Creativity. Washington, DC: The National Academies Press. doi: 10.17226/10671.
×

other devices; the future is likely to see many other devices emerge (e.g., music, medical, and kitchen appliances) that will be directly connected to the Internet, opening up many intriguing possibilities for ITCP work.30 Of course, traditional media, such as television, are also expected to evolve, partly in response to the Internet and partly on their own.31 Because the Internet is a transnational system, albeit with distributed management, it offers the potential of a worldwide forum. Global digital telecommunication establishes new ways of aggregating expertise and accumulating creative capital, thereby allowing people with similar interests to communicate with each other and share resources.

The World Wide Web, which served to popularize the Internet beginning in the 1990s, has an obvious attraction: It allows media to be distributed at low cost without any special organizational support, and the content can be viewed from anywhere in the world. The Web is quite flexible, offering the capability to publish text, images, sound, and video and to organize the presentation of material in creative ways. In the literary arts, for example, the combination of increasingly powerful desktop computers and the development of the Web has created unparalleled opportunities for people to engage in collaborative work at sites where, for example, hundreds of novice writers may contribute to a narrative. Such opportunities have led to an increasing amount of creative literature on the Web. That said, there are fundamental aspects of the design of the Web that constrain creative work. For example, the client/server model limits interactions either to following links or to performing computations at the server. In addition, the Web browser’s distinctive interface containing back and forward buttons works well for some types of work, but not for others, such as work trying to convey complex ideas—another example of the importance of the tradeoffs related to imposed structure. Given that a certain kind of creativity is inspired when faced with limitations, it may be that the limitations of the format can provide a framework for creative work, much as the traditional form of the classical symphony did for classical and romantic music, acting as a structure to both work within and rebel against.

Other models of Web programs suitable for ITCP work are possible. It is not that difficult to program networked applications that

30  

This paragraph is drawn primarily from Computer Science and Telecommunications Board, National Research Council, 2001, The Internet’s Coming of Age, National Academy Press, Washington, D.C.

31  

For example, new roles for public broadcasting (and for broadcasting more generally) may become possible. Television stations in the United States are required by law to convert to digital transmission and program production by December 2006. Initially, it may well be the case that programs are simply converted to digital format in toto. However, the creative integration of digital technology and content may enable many new possibilities for ITCP work. See Lawrence K. Grossman and Newton N. Minow, 2001, A Digital Gift to the Nation, Century Foundation Press, New York, available online at <http://www.digitalpromise.org>.

Suggested Citation:"3. Advancing Creative Practices through Information Technology." National Research Council. 2003. Beyond Productivity: Information Technology, Innovation, and Creativity. Washington, DC: The National Academies Press. doi: 10.17226/10671.
×

operate away from the biases and constraints of back and forward buttons and that are not subject to the biases and constraints of Hypertext Markup Language (HTML) and Flash (a multimedia plugin). More computational power is generally available locally, so highly interactive systems such as computer games run on a personal computer. Multiuser games can combine communication over the Internet with local computation. One could even imagine hybrid systems using cable TV, satellite, or telephone communication channels combined with the Internet and local computation. The development of grid infrastructures and applications32 provides additional capabilities for distributing and sharing capacity and activity, as do two-way protocols like those sponsored by the Universal Description, Discovery, and Integration (UDDI) project.33 The client/server model is evolving into a many-to-many, dynamic infrastructure.

Recent experience with Napster and other peer-to-peer systems has motivated experimentation among researchers and other creative communities in uses of the Internet for producing and distributing work. Peer-to-peer computing holds enormous promise for leveraging the intellectual and cultural resources of millions of people by allowing connections to be developed independent of centralized servers.34 Significant proposals include those from scientific researchers, such as Thomas Ray working in the field of artificial life, for research projects to run in the background on computers that are otherwise underused, for example, during the nighttime hours at most large organizations. Notwithstanding constraints on Napster, projects that do not involve copyright issues can still take advantage of peer-to-peer software to carry out research, share information, and share computing resources.35 Technologies such as peer-to-peer networking also can challenge basic notions of exhibition or participation.36 Multisite performances in games like Everquest, with persistent virtual realities that endure over days and months, allow participants to create complex virtual societies and emergent dramas. How might conventional museum exhibitions incorporate (or complement or compete with) such new applications of IT?

32  

A grid infrastructure is hardware and software infrastructure that supports wide-scale distributed computing, which enables high-performance applications.

33  

The project creates a platform-independent, open framework for describing services, discovering businesses, and integrating business services using the Internet, as well as an operational registry, according to the project Web site at <http://www.uddi.org>.

34  

Peer-to-peer networking as a technology is not so new per se. The growth of the number of Internet users across diverse segments of society provides the basis for new applications for ITCP work.

35  

Peer-to-Peer: Harnessing the Power of Disruptive Technologies (O’Reilly, Beijing and Cambridge, Mass., 2001), edited by Andy Oram, gives a good overview of these possibilities.

36  

Other efforts that may benefit from peer-to-peer computing include scholarly research to which amateurs have made significant contributions, for example in wide-ranging areas such as meteoritics (the study of meteors and mapping of the strewn fields of meteorites), Civil War history, and biography.

Technologies such as peer-to-peer networking can challenge basic notions of exhibition or participation.

Suggested Citation:"3. Advancing Creative Practices through Information Technology." National Research Council. 2003. Beyond Productivity: Information Technology, Innovation, and Creativity. Washington, DC: The National Academies Press. doi: 10.17226/10671.
×

The question is how creative practice can really leverage the capability of the Internet—going beyond pretty pictures on Web sites to address the real issues and opportunities that lie on the other side of the (fire)wall. The Listening Post project (which characterizes Internet traffic using sound) described in Chapter 2 represents one of the few ITCP initiatives that delve into the IT infrastructure. As Napster, SETI@Home, ICQ,37 and countless multiplayer games have proven, people—even millions of people—will take 5 or 10 minutes to download an application if it is sufficiently compelling. There are many opportunities for further work here.

But the commercialization of the Web and associated business and product developments place pressures on Web technology. For example, a break with earlier versions of a software package means that the authors of creative works must spend literally thousands of hours retrofitting those works when a few changes would have made it possible to avoid this tedious labor. This situation is similar to others where backwards compatibility has not been maintained; the larger issue of how vendors choose what features to add or drop reflects what economists refer to as market power, which influences the degree to which consumers have an effective voice. Although other groups of users would also have been inconvenienced by disruptive product changes, that artists and designers complain is a reflection of their comparative lack of financial and technical-support resources to help in adapting to change. A contrasting example of best practices in this context is Rob Kendall’s development of HTML authoring software, done in close cooperation with the people who are most likely to use it.38 See the section “Standards” below for a discussion of standards setting.

Given their roles as content-generators, artists and designers have a special interest in the Internet as a vehicle for content. The Web offers ready access to stored documents, access to which has often required a physical presence (e.g., to view tax rolls, government archives, materials in libraries, and the like), but these materials can increasingly be accessed easily by any Web publisher.39 The ability of virtually anyone to publish on the Web enables producers of ITCP work to contact audiences more directly. User-provided content plays an important role in the computing and Internet culture, thus offering a unique

37  

ICQ (pronounced as “I seek you”) is an online instant messaging program developed by Mirabilis Ltd. that is similar to AOL Time Warner’s Buddy List and Instant Messenger programs.

38  

See the Word Circuits Connection Muse, available online at <http://www.wordcircuits.com/connect/index.html>.

39  

The electronic documents on the Web are formatted in special languages (e.g., HTML) that accommodate multiple media (text, graphics, audio, video), enabling the development of creative applications ranging from online games to virtual museums and galleries. These documents also can be linked to other pages so that users can easily access more information.

Suggested Citation:"3. Advancing Creative Practices through Information Technology." National Research Council. 2003. Beyond Productivity: Information Technology, Innovation, and Creativity. Washington, DC: The National Academies Press. doi: 10.17226/10671.
×

market for innovators. A problematic issue with regard to Web content is how to manage it all. How does one assess and even cope with the amount of information available? How does one access what one wants? How do people know when information is valid? Will people rely on information retrieved for them, or will they still want to develop internal knowledge to be used for judgment? How does one redefine and control search engines? How do people ensure that search engines are objective without being trapped by gluey sites? How do people filter the results of searches to suit personal needs? New cultural models of editorial sense-making have been defined technically and to a certain degree tested: for example, collaborative filtering, “slash-dot”-style social intelligence applied to a common editorial base by combining a group rating, the social status of content-evaluators, and user interface design. These approaches are good examples of hybrids requiring strong arts and humanities skills in the content developing, shaping, rating, expression, and design, and strong IT to deliver effective new platforms sensitive to the community-content co-evolution.40

The engagement of artists and designers in the design discussions for new technologies can help ensure that the technologies are as powerful and thoughtful as possible—which is in the interest of everyone (see Box 3.6 for one possibility). However, engagement is a two-way street: Technologists need to be receptive to the input from artists and designers, and artists and designers must take the initiative to become engaged with technologists (e.g., by participating in mainstream technical conferences—and not only those related to animation or graphics).

ECONOMIC REALITIES

A key factor that shapes the development of tools for creative work is the economics of software. Software is expensive to develop and inexpensive to distribute. Therefore, it is most profitable when it appeals to many people, especially to non-experts (including the mass market), and competition places a premium on being first to market, another factor driving design compromises. The widespread use of a software application can provide a return on the investment required to create complex software systems. Of course, many hardware and software tools developed initially for niche groups eventually find their way into much broader, usually unanticipated, markets. Indeed, some of the best-known information technologies—from the spreadsheet to the Web—were designed initially for narrow purposes. But niche groups, including artists, have little luck in demanding special capabilities—even when they have good connections to vendors. For

40  

There is a well-developed, relevant literature in information retrieval, information filtering, information seeking, information organization, and related areas.

Software is expensive to develop and inexpensive to distribute. It is most profitable when it appeals to many people, especially to non-experts.

Suggested Citation:"3. Advancing Creative Practices through Information Technology." National Research Council. 2003. Beyond Productivity: Information Technology, Innovation, and Creativity. Washington, DC: The National Academies Press. doi: 10.17226/10671.
×

BOX 3.6 Databases and Artists

Databases touch nearly every aspect of daily life. They present opportunities that artists are only beginning to explore.1 Countless activities, such as using a credit card, visiting a physician, withdrawing cash from a bank, or accessing Web sites, involve a database that mediates the transaction. The operation of these databases—how (your) information flows—is largely invisible to the general public. One function that art can perform extraordinarily well is to make things visible. A cookie, for instance, is merely data flowing between computers. Imagine how an artist might be able to depict this flow to communicate to the general population what exactly is being transferred—and possibly even some of the implications of the transfer.

FIGURE 3.6.1 An artist’s perspective on database privacy. Illustration created by Jennifer M. Bishop, Computer Science and Telecommunications Board staff.

1  

See Victoria Vesna, 2002, “Database Aesthetics: Of Containers, Chronofiles, Time Capsules, Xanadu, Alexandria and the World Brain,” AI and Society, October 29, available online at <http://time.arts.ucla.edu/AI_Society/vesna_essay.html>.

Suggested Citation:"3. Advancing Creative Practices through Information Technology." National Research Council. 2003. Beyond Productivity: Information Technology, Innovation, and Creativity. Washington, DC: The National Academies Press. doi: 10.17226/10671.
×

example, a leading photographer who has achieved acclaim in digital and conventional photography has spent years working with corporations such as Adobe, Apple, Canon, Epson, and Kodak but reports that there are limits to what the software engineers understand about his artistic needs. He attends seminars with vendor engineers and asks for features he needs for his art making. He teases that the problem might be too hard for the engineers, hoping to harness their pride, recognizing that his goal is to get a new feature that is needed by only a very few users.41

Non-profit institutions and cooperative groups develop some of the best tools for creative use—both in computer science and in artistic and design contexts. This is true to a large extent because this software is simply not viable commercially and does not appear in that sector. Many good tools are non-commercial and open-source (see Chapter 7), developed and maintained by a number of programmers who coordinate their work through the Internet. This approach allows users to contribute capability directly, makes the software freely available, and allows users access to the inner workings of the software so that creative modifications and customization are possible. See Box 3.7. But it is not a panacea: Open-source tools tend to depend on volunteer efforts and often reflect the design effort of one or just a few primary implementers. As with closed-development/proprietary commercial products, users may not always get the features or design changes they need. Software robustness, documentation, and support may not reach commercial levels.42 Support and documentation tend both to benefit from and to attract a larger user community. Small, specialized projects often have the feel of “work in progress.”

Subsidies are a way to encourage the development of creative tools. This approach is exemplified by the Studio for Electro-Instrumental Music (STEIM), a center in Amsterdam that helps many musicians create new instruments using sensors.43 By subsidizing the engineering costs of sensor systems, STEIM has enabled the exploration of many novel electronic musical instrument designs. The engineering resources established by STEIM have attracted artists from around the world who otherwise would have no access to this technology. This approach seems to resemble that seen in computational science in the 1980s, when centers providing access to specialized computing resources were created. More generally, government (or philanthropic) support for research, including the development of prototypes and testbeds (see Chapter 8), is a way of subsidizing the development of creative tools that the market, all other things being equal, is not likely to develop in a timely manner. Whether induced through market

41  

Personal communication between the photographer and a reviewer, summer 2002.

42  

There are important counter-examples such as the Linux operating system, which is often used in critical server applications.

43  

For further information, see <http://www.steim.org/steim>.

Non-profit institutions and cooperative groups develop some of the best tools for creative use.

Suggested Citation:"3. Advancing Creative Practices through Information Technology." National Research Council. 2003. Beyond Productivity: Information Technology, Innovation, and Creativity. Washington, DC: The National Academies Press. doi: 10.17226/10671.
×

BOX 3.7 Open-Source Tools

Tools That Support ITCP Work

Python is a programming language created in the early 1990s by Guido van Rossum, who is still its principal author. Python has gathered a large following, finding important applications in graphics including Alice (see below), gaming, Web services, teaching, collaboration support, and research. Python is copyrighted but is free for use and distribution, and versions are owned by the Centrum voor Wiskunde en Informatica (National Research Institute for Mathematics and Computer Science in the Netherlands), the Corporation for National Research Initiatives, and BeOpen.com. In 2001, Zope Corporation and others sponsored the formation of the Python Software Foundation, a non-profit corporation set up to own the intellectual property of the most recent versions of Python. Conferences and workshops devoted to Python have been held since 1994, and in 2002, Python conferences were held in both the United States and Europe. Python users actively develop extensions and support for Python, including graphical programming environments and debugging tools, a re-implementation in Java (another popular programming language), and many new modules that support such things as Internet access, the XML language, linear algebra, encryption, and multimedia.1

PortAudio is an example of a much smaller open-software effort. PortAudio is a cross-platform library that supports audio input and output. It began with discussions about the need for such a library on an Internet mailing list, music-dsp.2 Hosted by the California Institute of the Arts, PortAudio was developed primarily by Ross Bencina and Phil Burk, who jointly control its design and distribution. More than 30 contributors have fixed bugs, offered expertise on the use of specific systems and hardware, and adapted PortAudio to new systems and devices. A dozen or so commercial and non-commercial applications are implemented using PortAudio.3

In both cases, it is interesting that many of the developers and contributors have never met face-to-face. Designs and work plans are discussed via e-mail and the community is informed of recent developments,

1  

See <http://www.python.org> for more information.

2  

See <http://shoko.calarts.edu/~glmrboy/musicdsp/whatis.html>.

3  

See <http://www.portaudio.com> for more information.

forces or supported by other resources, the research in directions discussed here for hardware and software can be a powerful investment in the future.

STANDARDS

A review of hardware and software would be incomplete without touching on the perennial issue of standards, which can both limit and support creativity. They obviously establish some constraints, but they also can allow various programs to interoperate in new and creative ways, and they are typically associated with stabilizing a market, lowering costs, and facilitating training.44 Although perhaps

44  

This situation is exemplified by consumer electronics, but it also applies to various forms of IT (and non-IT products).

Suggested Citation:"3. Advancing Creative Practices through Information Technology." National Research Council. 2003. Beyond Productivity: Information Technology, Innovation, and Creativity. Washington, DC: The National Academies Press. doi: 10.17226/10671.
×

problems, and solutions through electronic bulletin boards. Most of the contributors work without monetary compensation. In both examples, it is important to have a relatively small group to coordinate and integrate various contributions in order to maintain a coherent product.

Open-Source Tools for Learning

Python, based on early Lisp-like languages, is excellent for art and design education because of its simple and powerful syntax. An example of looking ahead to new models for languages is the Design by Numbers project at the Massachusetts Institute of Technology,4 which includes a simple language for graphical expression (called DBN), distributed freely over the Web, as a means to teach programming to visual artists and designers. This system is currently in use in more than 25 schools worldwide, with a sequel called the Proce55ing project that introduces two-dimensional and three-dimensional graphics concepts to visual designers.5

Individual tools available online include Alice,6 a three-dimensional (3D) interactive graphics programming environment for Windows built as a public-service project by a research group at Carnegie Mellon University. The intent was to make it easy for novices to develop interesting 3D environments and to explore the new medium of interactive 3D graphics. Alice is primarily a scripting and prototyping environment for 3D object behavior, not a 3D modeler, but it does read many common 3D file formats. By writing simple scripts, Alice users can control object appearance and behavior, and while the scripts are executing, objects respond to user input via mouse and keyboard. The current version of the Alice authoring tool is free to everyone and runs on computers that are commonly available for reasonable prices. Worlds created in Alice can be viewed and interacted with inside a standard Web browser once the Alice plug-in has been installed. The Alice core distribution includes a large library of textured models. The Alice plug-in does not allow authoring, but it does allow viewing of any Alice world.

4  

Led by John Maeda at the MIT Media Lab. See <http://dbn.media.mit.edu>.

5  

Proce55ing, an electronic sketchbook for developing ideas and a context for learning fundamentals of computer programming within the context of the electronic arts, was initiated by Ben Fry and Casey Reas at the MIT Media Lab and by the Interaction Design Institute Ivrea. As of November 2002, the software is in a pre-release stage, but bug fixes are being made in preparation for a more complete 1.0 release. Proce55ing will be free to download and use. See <http://proce55ing.net/>.

6  

See <http://www.alice.org>.

more obvious in IT, standards are common in arts and design arenas, from the dimensions of a violin to the Pantone color system. Standards can emerge from a standards-setting body or de facto from a leading supplier (as a consequence of competition within the marketplace). A good example for ITCP is digital images and associated software. Digital images are basically just arrays of pixels, yet they can be produced, stored, and manipulated in many ways. With standard representations for digital images, many programs and devices can interoperate, including cameras, digital editors, Web browsers, optical character reading (or other scanning) software, and graphical interface tools. But what should be standard? An image can exist in several forms—a set of high-level instructions for rendering, a set of polygons, or an array of pixels. A pixel representation may be a universal standard, but it forces the loss of high-level information. A high-level standard may be more complex to define, and to agree to, but it can

Suggested Citation:"3. Advancing Creative Practices through Information Technology." National Research Council. 2003. Beyond Productivity: Information Technology, Innovation, and Creativity. Washington, DC: The National Academies Press. doi: 10.17226/10671.
×

preserve the ability to perform high-level operations. Another example is the musical instrument digital interface (MIDI) standard. MIDI was created to connect music keyboards to synthesizers, but it was soon realized that MIDI could allow computers to control synthesizers. The availability of low-cost synthesizers that are more or less interchangeable has been critical to any number of creative computer music systems.

The rise of ITCP raises questions about the standards-setting process and the opportunities for artists and designers, as users, to have input. It is not uncommon for user groups to be on the sidelines of vendor efforts to set standards; artists and designers are not being singled out. And as recent attempts by non-profit organizations to explore how to inject “public interest” voices into IT standards-setting demonstrate, it is not easy for people with less technical depth, fewer financial resources, and different conversational norms to participate effectively in various IT standards-setting cultures.45 That said, a constructive step would be for specific communities of practice within the arts-design/ITCP world to try to determine for themselves a consensus view of the kinds of features or qualities they would like to see. A consensus statement would carry more weight with standards setters than would input from lone individuals or representatives of small or niche groups. As suggested above, research prototypes could aid in fostering consensus by demonstrating what a standard may lead to in practice.

SELECTED AREAS FOR THE DEVELOPMENT OF HARDWARE AND SOFTWARE THAT WOULD PROMOTE CREATIVE WORK

Precisely what the future holds is uncertain, but based on current expectations and trends,46 the capabilities available for work in ITCP will become far more powerful and diverse in the coming years. The timing of the present report is therefore propitious. Among notable research trends, nanotechnology is becoming a reality.47 Computers,

45  

For example, the advocacy group, the Center for Democracy and Technology, has tried an experiment in the last couple of years to see how it might participate in Internet Engineering Task Force (IETF) standards-setting activities (John Morris, a lawyer, was the group’s participant). The Ford Foundation also fostered discussion of standards setting in its Digital Media Forum. Of course, since the early days of computing there have been various user groups that tried to influence the product designs of major vendors.

46  

Raw computational power has been rising exponentially for years. A popular yardstick is Moore’s law: For the last 40 years, computing capability per dollar has doubled every 18 to 24 months, equivalent to a 100-fold improvement every 10 to 13 years, reflected in both rapidly increasing performance and declining price. Internet traffic has been growing by a factor of about two annually, outstripping the growth in computing speed.

47  

Nanotechnologies are generally defined as having structures in the size range of 1 to 100 nanometers (billionths of a meter). Technologies of this size often exhibit novel properties. See, for example, information on the National Nanotechnology Initiative at <http://www.nano.gov/>.

Research prototypes could aid in fostering consensus by demonstrating what a standard may lead to in practice.

Suggested Citation:"3. Advancing Creative Practices through Information Technology." National Research Council. 2003. Beyond Productivity: Information Technology, Innovation, and Creativity. Washington, DC: The National Academies Press. doi: 10.17226/10671.
×

sensors, and other devices are becoming ever smaller. New sensors for sound and light combined with faster and cheaper digital signal processors will make large-scale system sensing increasingly practical.48 The synthesis of materials to meet defined requirements means that the manufacture of devices will be simplified. In the near future, a number of specific changes are envisioned. Computing speed will become so great (10 gigahertz in a handheld device) that operations as complex as real-time voice recognition without a learning curve could be commonplace. There is also likely to be enhanced machine intelligence and capability for seeing, hearing, and speaking.49 And many devices will likely be considered disposable.

Thus, although some excellent tools for ITCP work are available, there are many opportunities for improvement. The committee identified a handful of areas that could exert considerable leverage in promoting ITCP, because such tools often enable the widespread adoption of new technologies.

Distributed Control

One area in need of support is distributed control. Software that coordinates multiple computers is among the most difficult of programs to write. This is true even for commercial business systems, but the problem is more acute in creative contexts because programming resources tend to be much more limited there. Tools can provide simplified protocols for communication between networked computers, including wireless devices.

The essentially sequential nature of software systems has made it difficult to envision a plausible distributed control paradigm. Much research in the related area of parallel computation was curtailed in the 1990s due to the evolution of the conventional microprocessor from the megahertz to gigahertz speed range in only a decade—there was no need to make computer programs run faster when the computer itself had become faster in a matter of months. Distributed control has taken center stage once again, not as a potential means to increase the speed of computation but as a way to handle the difficult task of coordinating computation across multiple computing units. There is a simple impediment to advancing the state of the art in this area; namely, programming methodologies for asynchronous computing are not in place, even in high-end systems. Software system designs inherit a legacy similar to that of the automobile industry in that only incremental, evolutionary changes are adopted instead of the kind of revolutionary advancements needed to write more sophis

48  

From Computer Science and Telecommunications Board, National Research Council, 2001, Embedded, Everywhere: A Research Agenda for Networked Systems of Embedded Computers, National Academy Press, Washington, D.C.

49  

See, for example, John Markoff, 2002, “Technology Gives Sight to Machines, Inexpensively,” New York Times, June 17, available online at <http://www.nytimes.com/2002/06/17/technology/17VISI.html>.

Suggested Citation:"3. Advancing Creative Practices through Information Technology." National Research Council. 2003. Beyond Productivity: Information Technology, Innovation, and Creativity. Washington, DC: The National Academies Press. doi: 10.17226/10671.
×

ticated programs that take advantage of multiprocessor and multisystem interactions.

Sensors and Actuators

Another area deserving attention is sensors and actuators. Artists and designers and scientists alike often want to collect data in the real world and to control mechanical devices using computers. Advanced sensing technologies and actuators exist, but without tools to simplify their use, these technologies remain out of reach of most individuals who are not specialists. Sensors and actuators that can be used by non-experts can promote creative endeavors (see, for example, Box 3.8). Note that the area of motion control and robotics is experiencing a renaissance in computer science: Advances in component technologies and artificial intelligence moved the field along considerably in the 1990s, and research in the area is expected to thrive.

A subtle reason for recent advancements in this area can be associated with the development of the Web. Once-difficult-to-obtain catalogs or obscure parts listings that were available only to hacker communities are now easily found on the Web with a quick search. Interfacing with these parts was once also difficult, requiring access to the relevant technical community. Now most of this knowledge is published and easily accessed on the Web. Interfacing information, sample projects, and even advanced research can easily be found by the curious builder.

In addition to the challenge of using sensor and actuator technologies, there is the problem of interfacing them to computers, especially if they are non-standard. This situation might be alleviated by the design and distribution of software and hardware that take advantage of serial interfaces such as the universal serial bus (USB) and the IEEE-1394 multimedia standard.50 There have been several attempts at developing general-purpose sensor systems and input/output boxes in the arts, such as the iCube system and notably, Michael Rodemer’s EZIO board, designed at the Art and Technology Department of the Art Institute of Chicago in the mid-1990s and now commercially available.

The success of the Parallax Inc. Basic Stamp microcontroller as a low-cost platform for educational experimentation has changed the public’s access to embedded computing. Ample documentation, complete technical information, and well-documented sample projects have

50  

One interesting effort is the development of “phidgets,” or physical widgets, at the University of Calgary. Phidgets are building blocks that assist in the development of physical user interfaces. Phidgets consist of a hardware device, software architecture for communication and connection management, a well-defined software application programming interface (API) for device programming, a simulation capability, and an optional component for interacting with the device, as described in “An Overview of Phidgets,” available online at <http://www.cpsc.ucalgary.ca/grouplab/phidgets/>.

Suggested Citation:"3. Advancing Creative Practices through Information Technology." National Research Council. 2003. Beyond Productivity: Information Technology, Innovation, and Creativity. Washington, DC: The National Academies Press. doi: 10.17226/10671.
×

made this the platform of choice for many budding “physical computing” courses nationwide. Once one is versed in the Basic Stamp microcontroller, it is easy to move to more economical systems by going one level deeper into the inner workings of the Basic Stamp to its central controller, the “PIC” chip. A wide variety of PIC chips, and competitor single-chip computers such as AVR Inc.’s line, have made it possible for anyone to easily build a sub-$10 computer for embedded control. Any art education program should find it attractive that platforms priced at less than $10 can be budgeted for as easily as any other art supply like paint, canvas, or a brush.

With respect to interfacing the various devices that are emerging today, one challenge is the speed and reliability of data transfer. Recently buses such as the CAN bus have emerged as high-speed standards for the robotics community, and the consumer-class USB and IEEE-1394 technologies are continually giving rise to faster versions like USB-2 and Firewire-2.51 Yet at some point the rapid progress in these standards can lead to confusion in the general population. What economists call the cost of switching—having to relearn a technology that one had finally felt comfortable with but then learned had suddenly become passé—can deter or slow adoption of technology, as well as influence standards-setting tactics.

Video and Audio

Extensive editing and processing tools exist for time-based media (including video and audio), but these tools assume a particular style of working—namely, that the product will be a linear video or audio recording. In addition, most tools do not support real-time processing. These limitations hamper artists who want to work with video and audio media in interactive performances, take other approaches to interactivity, stream material over the Internet, or use scripts to process or generate video.

Standards continually emerge and compete in this arena such as Apple QuickTime, Macromedia Flash, and Real Video. A common difficulty for all of these standards is to maintain compatibility across the multiple platforms they support. Often the same version of any system, such as Flash, will display a file properly on a PC but not on a Macintosh. These inconsistencies are rarely addressed, and content creators must simply work around these known problems. The standards are constantly upgraded, solving some problems yet introducing others. For an emerging field, like digital media or ITCP generally, standards that lack reliability are a serious problem. It is as if the art of film were emerging in an era when the film you might have shown just once might suddenly dissolve before your very eyes.

51  

Buses enable data transfer among the different components of a computer.

Suggested Citation:"3. Advancing Creative Practices through Information Technology." National Research Council. 2003. Beyond Productivity: Information Technology, Innovation, and Creativity. Washington, DC: The National Academies Press. doi: 10.17226/10671.
×

BOX 3.8 Ghostcatching

The fruit of a collaboration between multimedia artists Paul Kaiser and Shelley Eshkar and dancer/ choreographer Bill T. Jones, Ghostcatching1 is a virtual dance performance that makes use of motion-capture technology. Envisioning a blend of performance, filmmaking, drawing, and computer composition, the artists used light-sensitive sensors attached to 22 key points of Jones’s body and eight cameras to capture his movements. Roughly 40 sequences of Jones’s movement were recorded digitally. The resulting “capture data”—which represented a record of only the movements of the sensors and not of Jones’s body per se— were then used as the raw information for Kaiser and Eshkar’s creative application. See Figure 3.8.1. Jones’s “movements were then manipulated electronically and re-choreographed on a computer screen to make an original virtual performance.”2

The results include an 8-minute digital projection, 13 still images taken from the dance, photographs that describe the artists’ process, and a soundtrack consisting of Jones’s own sounds: “chanting, humming, singing, talking, grunting, and more.”3 Commissioned by the Cooper Union for the Advancement of Science and Art,4 the installation opened at the Cooper Union’s Arthur A. Houghton Jr. Gallery in New York City and ran from January 6 to February 13, 1999.

1. Improvising 2. Wearing motion-capture markers

1  

For more information, see <http://www.cooper.edu/art/ghostcatching/ghost/exhibit.html>.

2  

Zoe Ingalls, 1999, “Using New Technology to Create ‘Virtual Dance’,” Chronicle of Higher Education 45(21): A29-A30.

3  

Ingalls, 1999, “Using New Technology to Create ‘Virtual Dance’.”

Suggested Citation:"3. Advancing Creative Practices through Information Technology." National Research Council. 2003. Beyond Productivity: Information Technology, Innovation, and Creativity. Washington, DC: The National Academies Press. doi: 10.17226/10671.
×

3. Markers optically recorded and converted to digital three-dimensional files

4. Motion files applied to kinematic model of the body

5. “Hand-drawn” lines modeled as mathematical curves

6. Sampled charcoaled strokes applied and rendered as a final drawn body

FIGURE 3.8.1 Ghostcatching. Images courtesy of Bill T. Jones, Paul Kaiser, and Shelley Eshkar; available for download at <ftp://ftp.eshkar.com/ghostcatching/>.

Suggested Citation:"3. Advancing Creative Practices through Information Technology." National Research Council. 2003. Beyond Productivity: Information Technology, Innovation, and Creativity. Washington, DC: The National Academies Press. doi: 10.17226/10671.
×
Generative Processes

From both a computational and an artistic and design standpoint, an attractive feature of computers is that they enable generative processes. Rather then creating each note or brush stroke by hand, an artist or designer can design a process that generates material automatically. Of particular interest are fractals, chaotic systems, and systems exhibiting non-linear dynamics. These have strong connections to natural processes, can be explored only by using computers, and often create interesting and unpredicted output. A programmer could set up a generative process relatively easily, but tools are needed that can encapsulate these processes to make them more accessible to artists and designers. Generative processes are interesting to contemplate for a few reasons. First, the associated computer models depend on assumptions—and to ensure that the models do not present the kinds of constraints criticized above, it will be important for artists and designers to have some understanding of the models’ inner workings, which in turn depends on comfort with quantitative analyses. Second, some of the earliest explorations of ITCP were generative processes. A notable instance is Harold Cohen’s work with the artificial intelligence drawing-generating program, Aaron.52 More contemporary simulation models, some of which explore “artificial life,” have been used by biologists and other scientists to understand evolution, yielding a new understanding of its pacing and realization. The Santa Fe Institute led the development and use of these models, providing training in modeling methods, and it also has reached out to the arts community. The rise of computer games that feature artificial worlds suggests but one of the more obvious potential ITCP connections. Artificial-world development is also being explored outside the game context.53

When computing was young, the most natural form of creative activity on the computer was programming. This led to some landmark work by researchers at Bell Labs, such as Ken Knowlton, who were versed in programming but in addition had a penchant for visual art. The idea that forms can emerge from programming echoes complex processes in the physical world where the result is something you cannot expect. More recently with the rise and proliferation of digital tools, programmatic approaches to form have become less popular. For ITCP to advance, a fundamental shift is needed from computation or mathematics based on numbers to that based on symbols.

52  

See <http://www.umcs.maine.edu/~larry/latour/aaron.html>.

53  

See the World Generator/The Engine of Desire by Bill Seaman with Gideon May. The work enables the construction of virtual worlds in real time in an interactive environment. Although the work is artistic, it has been featured in a number of IT studies and has potential for many design and interactive tool applications. For further information, see <http://faculty.risd.edu/faculty/bseamanweb/web/texts.html> and <http://www.nada.kth.se/erena/pdf/D1_2.pdf>.

Suggested Citation:"3. Advancing Creative Practices through Information Technology." National Research Council. 2003. Beyond Productivity: Information Technology, Innovation, and Creativity. Washington, DC: The National Academies Press. doi: 10.17226/10671.
×
Reliable, Low-latency Communication over the Internet

Artists and designers have common cause with a wide range of users in their interest in reliable, low-latency communication over the Internet.54 Higher bandwidth and/or quality-of-service guarantees could enable new levels of interaction, distributed concerts, two-way video, and other creative activities that necessitate specific levels of network performance. As some observed to the committee, were artists able to stream their own bandwidth-demanding work out, it would yield greater diversity of voices and stimulation to creativity. This is the crux of the broadband dilemma: Broadband users are twice as likely as dial-up users to have their own Web sites, with nearly 60 percent becoming creators and managers of content in this manner, according to a report by the Pew Internet & American Life Project.55 But pervasive broadband deployment in the United States will require billions of dollars in investment;56 subscriber levels, while growing, fall far short of the potential, given existing deployment, and therefore widespread use is not expected soon. That said, in the academic world, the Internet2 project and complementary federal research infrastructure programs provide high-bandwidth service on and between many campuses because of its potential to support cutting-edge application research. Music performance over the Canadian equivalent, the CANARIE network, illustrates the general proposition that these facilities could be valuable ITCP vehicles. Recent advances in low-latency ultra-videoconferencing have been carried out by musicianresearchers at McGill and Stanford Universities.57

As in other communities, opinions in the arts world about the value of broadband differ. To some, a benefit of ever-increasing processor speeds has been the development of media-rich simulations and other works, stimulating demand for broadband services. Yet others find such digital spectacles less compelling than new social models for peer-to-peer cultural production. From this standpoint, advanced content should be focused not on creating volumes of traffic

54  

The Internet is based on best-effort policies that make no guarantees for the timely delivery of information. Reliability is achieved by re-transmitting data when a loss is detected, but this recovery introduces delays. See Computer Science and Telecommunications Board, National Research Council, 2001, The Internet’s Coming of Age, National Academy Press, Washington, D.C.

55  

Mike Snider, 2002, “Faster Connection Allows Users to Do More,USA Today, June 23, available online at <http://www.usatoday.com/life/cyber/tech/2002/06/24/broadband.htm>. See also The Broadband Difference: How Online Americans’ Behavior Changes with High-speed Internet Connections at Home, available online <http://www.pewinternet.org/reports/toc.asp?Report=63>.

56  

See Computer Science and Telecommunications Board, 2001, The Internet’s Coming of Age, p. 45.

57  

Jeremy Cooperstock, an engineer at McGill University, and Chris Chafe, a composer at Stanford University; see <http://www.cim.mcgill.ca/~jer/research/rtnm/>. Internet2 and CANARIE are discussed further in Chapters 5 and 8.

Suggested Citation:"3. Advancing Creative Practices through Information Technology." National Research Council. 2003. Beyond Productivity: Information Technology, Innovation, and Creativity. Washington, DC: The National Academies Press. doi: 10.17226/10671.
×

that can fill up fat data pipes, but on infrastructural advances that would allow a wide public access to and control over digital objects, although with the recognition that there are certainly legitimate and productive needs for greater capacity.58 This perspective shifts the focus to thinking of how to increase the range of interpersonal connection and personal expression distributed across society.

One issue relating to broadband is that the quantitative increase in bit traffic makes possible qualitative change in digital expression. However, this potential is confounded by a tendency for capabilities to fluctuate with IT innovations: After the initial emergence of digital media serving smaller amounts of kilobytes, higher amounts of data could be conveyed through the CD-ROM media, only to go back to smaller amounts of kilobytes over the (narrowband or dial-up) Web, and then up to large capacity again via DVD-ROM and now to broadband. Looking across this series of innovations, they have not been associated with an intrinsic change in content for the better—perhaps because of uncertainty associated with the fluctuations, or perhaps because experience with any one technology has been too abbreviated.

Tool Design and Human-Computer Interaction

There is a great deal of work to be done in understanding the ramifications of the tradeoffs that occur every day in the design of tools. The discussion above includes examples of how the tool designers’ choices in areas such as abstraction, extensibility, and metaphor can affect the practices within a field and the results obtained through the use of the tool. Insufficient work has gone into understanding how to leverage this relationship to better support creative practices. This goes far beyond the usability studies once typically associated with human-computer interaction research. Questions to be addressed include these: When does embodying domain knowledge in a tool help? What is the impact of supporting multiple metaphors in one tool? When are constraints good inspirations, and when do they cause excessive perspiration? When can implicit assumptions help, and when can they be hindrances? How much and what kind of openness yields opportunity, and what yields frustration? In other words, what is the relationship among information, its representation, presentation, and manipulation, and the range of work encouraged? Researchers and developers have produced a small number of data points—more work in understanding this area can have deep ramifications for encouraging ITCP.

58  

A Canadian task force concluded a series of consultations with the arts and cultural community with a report entitled Filling the Pipe: Stimulating Canada’s Broadband Content Industry Through R&D, available online at <http://www.canarie.ca/press/publications/broadband_report.pdf>.

A quantitative increase in bit traffic makes possible qualitative change in digital expression.

Suggested Citation:"3. Advancing Creative Practices through Information Technology." National Research Council. 2003. Beyond Productivity: Information Technology, Innovation, and Creativity. Washington, DC: The National Academies Press. doi: 10.17226/10671.
×
Programming Languages

Finally, some areas of great interest to artists are not well served by programming languages, which tend to be driven by science, engineering, the Web, and more commercial application areas. Some of the topics discussed above, especially distributed control, video and audio processing, and generative processes, could be supported better by languages. Languages are particularly important for specifying interactive behavior and generative processes, an area not well supported by commercial software. Because a great deal of creative activity involves combining existing concepts in new ways, and because programming languages provide the glue for assembling software tools and libraries into applications, languages are critical to innovation. Progress has been achieved in making programming languages simpler for novice programmers, but this work needs to be adapted to support more creative work.

Suggested Citation:"3. Advancing Creative Practices through Information Technology." National Research Council. 2003. Beyond Productivity: Information Technology, Innovation, and Creativity. Washington, DC: The National Academies Press. doi: 10.17226/10671.
×
Page 61
Suggested Citation:"3. Advancing Creative Practices through Information Technology." National Research Council. 2003. Beyond Productivity: Information Technology, Innovation, and Creativity. Washington, DC: The National Academies Press. doi: 10.17226/10671.
×
Page 62
Suggested Citation:"3. Advancing Creative Practices through Information Technology." National Research Council. 2003. Beyond Productivity: Information Technology, Innovation, and Creativity. Washington, DC: The National Academies Press. doi: 10.17226/10671.
×
Page 63
Suggested Citation:"3. Advancing Creative Practices through Information Technology." National Research Council. 2003. Beyond Productivity: Information Technology, Innovation, and Creativity. Washington, DC: The National Academies Press. doi: 10.17226/10671.
×
Page 64
Suggested Citation:"3. Advancing Creative Practices through Information Technology." National Research Council. 2003. Beyond Productivity: Information Technology, Innovation, and Creativity. Washington, DC: The National Academies Press. doi: 10.17226/10671.
×
Page 65
Suggested Citation:"3. Advancing Creative Practices through Information Technology." National Research Council. 2003. Beyond Productivity: Information Technology, Innovation, and Creativity. Washington, DC: The National Academies Press. doi: 10.17226/10671.
×
Page 66
Suggested Citation:"3. Advancing Creative Practices through Information Technology." National Research Council. 2003. Beyond Productivity: Information Technology, Innovation, and Creativity. Washington, DC: The National Academies Press. doi: 10.17226/10671.
×
Page 67
Suggested Citation:"3. Advancing Creative Practices through Information Technology." National Research Council. 2003. Beyond Productivity: Information Technology, Innovation, and Creativity. Washington, DC: The National Academies Press. doi: 10.17226/10671.
×
Page 68
Suggested Citation:"3. Advancing Creative Practices through Information Technology." National Research Council. 2003. Beyond Productivity: Information Technology, Innovation, and Creativity. Washington, DC: The National Academies Press. doi: 10.17226/10671.
×
Page 69
Suggested Citation:"3. Advancing Creative Practices through Information Technology." National Research Council. 2003. Beyond Productivity: Information Technology, Innovation, and Creativity. Washington, DC: The National Academies Press. doi: 10.17226/10671.
×
Page 70
Suggested Citation:"3. Advancing Creative Practices through Information Technology." National Research Council. 2003. Beyond Productivity: Information Technology, Innovation, and Creativity. Washington, DC: The National Academies Press. doi: 10.17226/10671.
×
Page 71
Suggested Citation:"3. Advancing Creative Practices through Information Technology." National Research Council. 2003. Beyond Productivity: Information Technology, Innovation, and Creativity. Washington, DC: The National Academies Press. doi: 10.17226/10671.
×
Page 72
Suggested Citation:"3. Advancing Creative Practices through Information Technology." National Research Council. 2003. Beyond Productivity: Information Technology, Innovation, and Creativity. Washington, DC: The National Academies Press. doi: 10.17226/10671.
×
Page 73
Suggested Citation:"3. Advancing Creative Practices through Information Technology." National Research Council. 2003. Beyond Productivity: Information Technology, Innovation, and Creativity. Washington, DC: The National Academies Press. doi: 10.17226/10671.
×
Page 74
Suggested Citation:"3. Advancing Creative Practices through Information Technology." National Research Council. 2003. Beyond Productivity: Information Technology, Innovation, and Creativity. Washington, DC: The National Academies Press. doi: 10.17226/10671.
×
Page 75
Suggested Citation:"3. Advancing Creative Practices through Information Technology." National Research Council. 2003. Beyond Productivity: Information Technology, Innovation, and Creativity. Washington, DC: The National Academies Press. doi: 10.17226/10671.
×
Page 76
Suggested Citation:"3. Advancing Creative Practices through Information Technology." National Research Council. 2003. Beyond Productivity: Information Technology, Innovation, and Creativity. Washington, DC: The National Academies Press. doi: 10.17226/10671.
×
Page 77
Suggested Citation:"3. Advancing Creative Practices through Information Technology." National Research Council. 2003. Beyond Productivity: Information Technology, Innovation, and Creativity. Washington, DC: The National Academies Press. doi: 10.17226/10671.
×
Page 78
Suggested Citation:"3. Advancing Creative Practices through Information Technology." National Research Council. 2003. Beyond Productivity: Information Technology, Innovation, and Creativity. Washington, DC: The National Academies Press. doi: 10.17226/10671.
×
Page 79
Suggested Citation:"3. Advancing Creative Practices through Information Technology." National Research Council. 2003. Beyond Productivity: Information Technology, Innovation, and Creativity. Washington, DC: The National Academies Press. doi: 10.17226/10671.
×
Page 80
Suggested Citation:"3. Advancing Creative Practices through Information Technology." National Research Council. 2003. Beyond Productivity: Information Technology, Innovation, and Creativity. Washington, DC: The National Academies Press. doi: 10.17226/10671.
×
Page 81
Suggested Citation:"3. Advancing Creative Practices through Information Technology." National Research Council. 2003. Beyond Productivity: Information Technology, Innovation, and Creativity. Washington, DC: The National Academies Press. doi: 10.17226/10671.
×
Page 82
Suggested Citation:"3. Advancing Creative Practices through Information Technology." National Research Council. 2003. Beyond Productivity: Information Technology, Innovation, and Creativity. Washington, DC: The National Academies Press. doi: 10.17226/10671.
×
Page 83
Suggested Citation:"3. Advancing Creative Practices through Information Technology." National Research Council. 2003. Beyond Productivity: Information Technology, Innovation, and Creativity. Washington, DC: The National Academies Press. doi: 10.17226/10671.
×
Page 84
Suggested Citation:"3. Advancing Creative Practices through Information Technology." National Research Council. 2003. Beyond Productivity: Information Technology, Innovation, and Creativity. Washington, DC: The National Academies Press. doi: 10.17226/10671.
×
Page 85
Suggested Citation:"3. Advancing Creative Practices through Information Technology." National Research Council. 2003. Beyond Productivity: Information Technology, Innovation, and Creativity. Washington, DC: The National Academies Press. doi: 10.17226/10671.
×
Page 86
Suggested Citation:"3. Advancing Creative Practices through Information Technology." National Research Council. 2003. Beyond Productivity: Information Technology, Innovation, and Creativity. Washington, DC: The National Academies Press. doi: 10.17226/10671.
×
Page 87
Suggested Citation:"3. Advancing Creative Practices through Information Technology." National Research Council. 2003. Beyond Productivity: Information Technology, Innovation, and Creativity. Washington, DC: The National Academies Press. doi: 10.17226/10671.
×
Page 88
Suggested Citation:"3. Advancing Creative Practices through Information Technology." National Research Council. 2003. Beyond Productivity: Information Technology, Innovation, and Creativity. Washington, DC: The National Academies Press. doi: 10.17226/10671.
×
Page 89
Suggested Citation:"3. Advancing Creative Practices through Information Technology." National Research Council. 2003. Beyond Productivity: Information Technology, Innovation, and Creativity. Washington, DC: The National Academies Press. doi: 10.17226/10671.
×
Page 90
Suggested Citation:"3. Advancing Creative Practices through Information Technology." National Research Council. 2003. Beyond Productivity: Information Technology, Innovation, and Creativity. Washington, DC: The National Academies Press. doi: 10.17226/10671.
×
Page 91
Suggested Citation:"3. Advancing Creative Practices through Information Technology." National Research Council. 2003. Beyond Productivity: Information Technology, Innovation, and Creativity. Washington, DC: The National Academies Press. doi: 10.17226/10671.
×
Page 92
Suggested Citation:"3. Advancing Creative Practices through Information Technology." National Research Council. 2003. Beyond Productivity: Information Technology, Innovation, and Creativity. Washington, DC: The National Academies Press. doi: 10.17226/10671.
×
Page 93
Suggested Citation:"3. Advancing Creative Practices through Information Technology." National Research Council. 2003. Beyond Productivity: Information Technology, Innovation, and Creativity. Washington, DC: The National Academies Press. doi: 10.17226/10671.
×
Page 94
Suggested Citation:"3. Advancing Creative Practices through Information Technology." National Research Council. 2003. Beyond Productivity: Information Technology, Innovation, and Creativity. Washington, DC: The National Academies Press. doi: 10.17226/10671.
×
Page 95
Next: 4. The Influence of Art and Design on Computer Science Research and Development »
Beyond Productivity: Information Technology, Innovation, and Creativity Get This Book
×
Buy Paperback | $48.00 Buy Ebook | $38.99
MyNAP members save 10% online.
Login or Register to save!
Download Free PDF

Computer science has drawn from and contributed to many disciplines and practices since it emerged as a field in the middle of the 20th century. Those interactions, in turn, have contributed to the evolution of information technology – new forms of computing and communications, and new applications – that continue to develop from the creative interactions between computer science and other fields.

Beyond Productivity argues that, at the beginning of the 21st century, information technology (IT) is forming a powerful alliance with creative practices in the arts and design to establish the exciting new, domain of information technology and creative practices—ITCP. There are major benefits to be gained from encouraging, supporting, and strategically investing in this domain.

  1. ×

    Welcome to OpenBook!

    You're looking at OpenBook, NAP.edu's online reading room since 1999. Based on feedback from you, our users, we've made some improvements that make it easier than ever to read thousands of publications on our website.

    Do you want to take a quick tour of the OpenBook's features?

    No Thanks Take a Tour »
  2. ×

    Show this book's table of contents, where you can jump to any chapter by name.

    « Back Next »
  3. ×

    ...or use these buttons to go back to the previous chapter or skip to the next one.

    « Back Next »
  4. ×

    Jump up to the previous page or down to the next one. Also, you can type in a page number and press Enter to go directly to that page in the book.

    « Back Next »
  5. ×

    Switch between the Original Pages, where you can read the report as it appeared in print, and Text Pages for the web version, where you can highlight and search the text.

    « Back Next »
  6. ×

    To search the entire text of this book, type in your search term here and press Enter.

    « Back Next »
  7. ×

    Share a link to this book page on your preferred social network or via email.

    « Back Next »
  8. ×

    View our suggested citation for this chapter.

    « Back Next »
  9. ×

    Ready to take your reading offline? Click here to buy this book in print or download it as a free PDF, if available.

    « Back Next »
Stay Connected!