Towards Interactive Aesthetics


'Myron W. Krueger Myron W. Krueger

A third of a century ago, computers were programmed with punched cards and even programmers were seldom permitted in the air-conditioned temples where they were hidden. It was the Age of the Impersonal Computer. Artists had begun to use computers, but primarily to generate visual content for traditional graphic media.

As a graduate student in computer science then at the University of Wisconsin, I thought of the encounter of Human and Machine as the central drama of our time and wanted to be on the front line. However, outside the world of the technology specialist, there was a profound distrust of computers. This fear was based on a number of misconceptions: that computers were inherently mathematical, that they were solely for practical purposes, and that they threatened our humanity. I wanted to try to change these attitudes by exposing people to a brief aesthetic experience that demonstrated that computers could be imbued with whimsy and need not be doomed to tedious problem solving and record keeping.

My first effort in this direction was a collaboration with Dan Sandin, who later developed the data glove and CAVE, as well as other artists, composers, and technologists on a computer controlled responsive environment called Glowflow. This project had limited provision for interactivity; however, the sculptor who designed the visual aspect of the piece did not feel that interactivity was consistent with the contemplative character of the work. My reaction to this argument was that interactivity was the one new dimension that it had been impossible to explore before the invention of the computer. Therefore, new pieces should be conceived with interactivity in mind. In fact, it would be necessary to elevate interactivity to the level of a medium in its own right rather than to try to make conventional art works interactive. Making interactivity central to the work would alter the definition of art and would change the relationships among artist, art work, and art consumer just as computer technology was already transforming other aspects of modern culture. By choosing to investigate interactivity itself, I set aside concerns about whether the resulting medium was art and instead examined only the technological and aesthetic dimensions that would be required to create the ultimate interactive experience. Only when the medium was developed would it be possible to ask whether people would interact with it and whether the resulting interactions could be viewed as art.

Any conversation between a participant and a piece would be constrained by what the computer knows about the participant’s actions and the responses that it can generate. For the computer to have the maximum expressive power, the visual displays would need to be completely programmable. Computer graphics was the only richly programmable visual medium that existed at the time. Although it was limited to white lines drawn on black CRT screens, I reasoned that if the scale could be greatly increased, the result would be an unbeautiful but authoritative display. Since data projectors did not yet exist, aiming a video camera at the computer monitor converted the vector graphic image into a signal that could be fed into a video projector. This may have been the first time a computer image was projected and was certainly the first time computer graphics were mixed with live video images to provide the basis for a computer-mediated, whole-body experience. In order for the computer to generate interesting responses which could be displayed to the participant, it would have to receive complex input from him. Rather than use what later became the traditional computer/human interface of a sedentary user aiming a pointing device, it was decided that the experience should be divorced from any recognizable user experience and instead should employ the body as the primary means of exploring this new reality. Thus, the computer should perceive the participant’s body and respond to his movements.

This decision meant that the symbolic communication, seen as the hallmark of human intelligence by the artificial intelligence community, would be bypassed in order to address the fundamental machinery of experience: the perception of the world and the manipulation of it through physical behavior. Whereas Ivan Sutherland said that the ultimate display would be a virtual room in which you could sit and be surrounded by computer graphics, I wanted to use my body to move around the graphic world and to manipulate the objects within it.[Suth65]

To give the computer information about participants’ behavior, it was necessary to think in terms of senses, not inputs. Two approaches were used. The first was a sensory floor comprising a thousand handmade pressure switches. The second was to give the computer a video-based form of vision which it would use to actually see the participants. Since real-time computer vision did not yet exist, I adopted an expedient I called the “Wizard of Oz Interface” in which I perceived the participants’ behavior, interpreted what I saw, and triggered the computer’s responses using a data tablet.

This technique was employed in an installation called Metaplay which was shown at the University of Wisconsin Union Gallery in 1970. Participants entered the installation and saw their live video images projected in front of them. Quickly graffiti appeared on their images. Initially, they were amused by the drawing but soon wanted to interact with the person who was doing the drawing. They hid from the cursor. When it got close, they batted it away. When the cursor was changed to a ball, they hit it and were delighted when it was moved across the screen as if propelled by their action. It was discovered that they could be taught to draw by moving their fingers in the air. All of these interactions were accomplished by my observing their actions and generating the graphic responses with a data tablet in the Computer Center a mile away.

By being in the feedback loop ten hours a day, seven days a week, for six weeks, I myself was forced to observe participants interacting with my piece and developed a strong intuition about what people could understand, what they were willing to do, and what they wanted to make happen. Since my ideas arose directly from interaction with participants rather than from preconceived notions, I escaped the pitfall of creating art that sounds impressive when described in writing, but that does not succeed with its audience. Shortly after the original installations, a sixteen year development of the computer vision capability that was need to make the work autonomous was begun. All subsequent work was guided by an interactive aesthetics learned in the early pieces:

1. Art should be playful and accessible.


2. Physical interaction is the new dimension that should be explored. Although our culture only permits adults limited modes of physical movement, interactive art can induce them to move in new ways.


3. Displays should dominate the participants’ field of view so that they are immersed in the interactive experience.


4. The participants’ image is a useful ingredient in the visual display. People consider their image an extension of their identity. What happens to it, happens to them. What touches it, they feel.


5. The computer’s responses should be obvious. There should always be a surface level which people can understand within a few seconds. In a world in which interactive experiences compete with many other alternatives, a piece which is not apprehended almost immediately will be quickly abandoned.


6. All computer-generated stimuli should be in response to participants’ actions. When the computer initiates gratuitous stimuli, the participants’ ability to understand what is happening is threatened.


7. Graphic realism is not necessary for compelling interaction. In fact, realistic graphic environments often contain clutter that makes it unclear what interactive possibilities the virtual world affords.
   
8. Interaction between the participant and the virtual world should be fluid and seamless. This means that the responses should be instantaneous, just as they would be in the real world. Just as a vehicle that can taxi down a runway but not move fast enough to take off is not an airplane, an interactive experience in which the participant is aware of a lag between his action and the piece’s response is not interactive.


9. Giving the audience many different interactions to choose from keeps them interested longer.
   
10. If offered a canvas, brush, and oil paints in a public setting, few people will attempt to create art, but when confronted with the opportunity to use a medium in which the rules are unknown and with which the chances of aesthetic success are high, most people will move their bodies to create a result that pleases them. Indeed, aesthetic exploration provides a whole new reason to move the body.
    
11. Ideally, a piece should do something that makes sense under all exhibit conditions: crowded versus empty, noisy versus silent, lone individuals versus small groups.
    
12. Whereas video games rely on scores to motivate participation in highly structured experiences, physical participation permits other styles of interaction to be developed.
    
13. Encumbering devices like head-mounted displays distance the participant from the virtual experience and are not yet good enough to use.