Music Creatures


'Marc Downie Marc Downie

This interactive installation consists of a colony of musical creatures that live within the sonic environment of the gallery. These autonomous, virtually embodied creatures have the ability to perceive, act and survive in musical worlds. They possess abstract, simple bodies, the control of which they learn, and the movement of which creates sound. Played out in a spatial representation, fragments of sound heard by the creatures are exchanged and distorted; retrograde variations and repeated sub-segments are swapped back and forth in an accidental, recombinant evolution; sounds from the environment are drawn into the knowledge that the creatures come to possess about their world and their bodies.

In this respect this audio-visual work does not present a mapping between sound and image, but rather draws parallels between human-musical problems and the problems of motor control and learning in animals, playing with the very physicality of the sound creation process. A dynamic, spatial arrangement of sonic elements allows us to recapture and recast many musical transformations—passages can be inverted, fused, or temporally manipulated. By transferring musical passages into space, musical problems are transformed into problems for these synthetic bodies. For these artificial intelligences do not dance reactively, but move because they are capable of predicting, and to some extent understanding, sound/body contingencies constructed from their own experience. Ultimately these creatures, who in their colony interact with each other, interact with us: we can use musical instruments to directly inject musical material into the colony or let the creatures work with the ambient sound in the gallery space. The adaptation and learning present in these creatures ensure that the colony maintains a position on an edge of musical complexity.

The work of the Synthetic Characters Group at the MIT Media Lab focuses on the creation of expressive, autonomous creatures in interactive installations. These creatures’ personalities and interactions are authored in high level terms: drives and motivations, goals and expectations. Our research is grounded on a number of years of work on the design of such systems, and inspired by the study of real animals. In this work we are driven by the desire to investigate not only the relationship between musical problems and motor problems but to engage the (necessarily) biological roots of human music and press the field of artificial intelligence into the service of interactive music and digital animation. In this wider context, we have a bold hypothesis: only by beginning with a study of the animal roots of musical behavior—roots which may include the organization of both sound and gesture in time—can we begin to create systems with which we can interact musically. We further hypothesize that a study of proto-musical capabilities and the commonalities of animals may ultimately prove more useful for the creation of new interactive music and in the construction of primitive, artificial, interactive musical agency than any conventional music theory. Therefore, we make perceptive learning and motor representations of these music creatures biologically plausible, in the hope that they will be useful for artistic ends.

Despite a recent resurgence of interest in such biomusicology, science cannot yet provide a computationally constructive or artistically useful theory of musical production, consumption or collaboration. And it is unlikely that purely biological approaches will bear fruit without complementary constructive artistic experimentation. These musical creatures are early moves towards these goals.

Marc Downie with the Synthetic Characters Group at the MIT Media Lab.