|
Blurring the Boundaries between Bodies, Bits and Atoms: From ClearBoard to Tangible Bits
'Hiroshi Ishii
Hiroshi Ishii
AT THE SEASHORE Where the land meets the sea, there is a border. This borderline is constantly moving and changing the shape of the shore. However, it is not only a simple line of a landscape, but also a critical biological locale. Whenever the tides go in and out, the inhabitants in the intertidal zone experience harsh environmental changes. When the tidal animals are exposed, they must withstand the heat and drying. When they are submerged, they must face the constant pounding of waves. Living at this boundary is hard, but it is also a fertile environment for many kinds of life and activity [Ten 87].THE DIGITAL SEASHORE Where bits meet atoms, there is another kind of shoreline. Today, we live on this boundary between physical space and cyberspace. The constant pounding by waves of bits distorts our experience of space, challenging the boundaries of our physical bodies.
Living on this edge is hard because our bodies remain in the physical world, even while we are immersed in digital information. However, this boundary marks a most interesting challenge for designers of media to bridge the gap between the worlds of bits and atoms.
People tend to think of borders as separators, but it is important to understand borders as "interfaces" which actively promote exchange between adjoining worlds. This presentation is about such kinds of interface design. To address the issue of designs which blur the boundary between our bodies, cyberspace, and physical space, I will present two media design projects: ClearBoard [1991–94] at NTT Human Interface Laboratories, and Tangible Bits [1996– present] at MIT Media Laboratory.
The goal of these projects is to go beyond currently dominant models of "talking head" and "painted bits" by releasing the information out of computers and projecting it into architectural spaces where our bodies reside. I will present our design approaches from the perspective of HCI [Human-Computer Interaction] and CSCW [Computer-Supported Cooperative Work] research.DISTORTION OF HYBRID SPACE Media technology extends the space we inhabit and within which we interact with others, but often in abrupt and arbitrary ways. These "hybrid spaces" can be distorted and discontinuous, with our bodies often losing orientation. These hybrid spaces are no longer seamless.
Video telephony creates interpersonal space that maintains a sense of "telepresence" or "copresence" through the visibility of gestures and facial expressions of remote people. However, the distortions of hybrid space make it difficult to utilize various nonverbal cues to interact smoothly and naturally with remote colleagues.
A limitation of current collaboration media is the ad hoc configuration of shared workspaces, typically consisting of groupware running in a window on a computer screen next to a small "talking head" window. This is like looking at a cubist painting in which space is fragmented. In these electronic shared workspaces, small mouse pointers are completely disembodied, and the discontinuity between the remote person's image and the shared workspace is amplified.CLEARBOARD AND SEAMLESS SPACE DESIGN ClearBoard was designed in collaboration with Minoru Kobayashi at NTT to realize a seamless integration of shared workspace and interpersonal space over a distance [Fig. 1]. A design goal of ClearBoard was to allow a pair of users to shift easily between interpersonal space and shared workspace using familiar everyday cues such as a partner's gestures, head movements, eye contact, and gaze direction. The key metaphor of ClearBoard design is talking through and drawing on a large transparent glass board [Ishii, Kobayashi, and Grudin 93].
Figure 2 shows ClearBoard-1, the first prototype to support remote collaboration. Two users are discussing a route by drawing a map directly on the screen surface. Both users can share a common map orientation by a mirror reversing the video images. The partners can read all the
text and graphics in their correct orientation.
We observed effortless switching of focus between the task and the partner's face. Users could read their partner's facial expression, achieve eye contact, and utilize awareness of the direction of their partner's gaze. We call it "gaze awareness": the ability to monitor the direction of a partner's gaze and thus his or her focus of attention.FROM CLEARBOARD TO TANGIBLE BITS The ClearBoard design not only goes beyond the traditional desktop metaphor and windows-based GUI, but also goes beyond the imitation of physical proximity [or "being there"] which has long been the unquestioned goal of telecommunication technologies. ClearBoard suggests a direction of "beyond being there" [Hollan 92]. Further, ClearBoard evolves the concept of the physical wall from a passive partition to a dynamic collaboration medium that integrates distributed real and virtual spaces.
ClearBoard inspired a vision of new architectural spaces where all surfaces including walls, ceilings, desktops, windows, and doors become "interactive surfaces" through which people can interact with other real and virtual environments [Ishii, Kobayashi, and Arita 94].
With Tangible Bits, we are working towards realizing this vision, coupled simultaneously with two new key ideas. First, we are interested in using interactive surfaces in combination with graspable physical objects used to physically embody and manipulate digital information. Secondly, we wish to couple interactive surfaces with "ambient media," using ambient light, sound, airflow, and even water movement to communicate information at the periphery of human awareness.
The interactions between people and cyberspace are now largely confined to traditional GUI [Graphical User Interface]-based boxes sitting on desktops or laptops. The interactions with these GUIs are separated from the ordinary physical environment within which we live and interact. Although we have developed various skills and work practices for processing information through haptic interactions with physical objects as well as peripheral senses, most of these practices are neglected in current HCI design because of the lack of diversity of input/ output media, and too much bias towards graphical output at the expense of input from the real world.KEY IDEAS OF TANGIBLE BITS We see the locus of computation now shifting from the desktop in two major directions: i] onto our skins/bodies, and ii] into the physical environments we inhabit. The transition to our bodies is represented by recent activities in the new field of "wearable computers" [Mann 96]. We are focusing on the second path: integration of computational augmentations into the physical environment. Our intention is to take advantage of natural physical affordances to achieve a heightened legibility and seamlessness of interaction between people and digital information.
Tangible Bits is an attempt to bridge the gap between cyberspace and the physical environment by making digital information [bits] tangible. We are developing ways to make bits accessible through the physical environment. Our key concepts are:- Interactive Surfaces: Transformation of each surface within architectural space [e.g., walls, desktops, ceilings, doors, windows] into an active interface between the physical and virtual worlds;
- Coupling of Bits and Atoms: Seamless coupling of everyday graspable objects [e.g., cards, books, models] with the digital information that pertains to them; and
- Ambient Media: Use of ambient media such as sound, light, airflow, and water movement for background interfaces with cyberspace at the periphery of human perception [Ishii and Ullmer 97].
Ultimately, we are seeking ways to turn each state of physical matter – not only solid matter, but also liquids and gases – within everyday architectural spaces into "interfaces" between people and digital information. We are exploring ways of both improving the quality and broadening the bandwidth of interaction between people and digital information by:- allowing users to "grasp & manipulate" foreground bits by coupling bits with physical objects, and
- enabling users to be aware of background bits at the periphery using ambient media in an augmented space.
Current HCI research is focusing primarily on foreground activity and neglecting the background.
However, subconsciously, people are constantly receiving varied information from the "periphery" without attending to it explicitly. If anything unusual is noticed, it immediately comes to the center of their attention.
The smooth transition of users' focus of attention between background and foreground using ambient media and graspable objects is a key challenge of Tangible Bits.
Our group designed several platforms including metaDESK, ambientROOM [Ishii and Ullmer 97], inTouch [Brave and Dahley 97], and Triangles [Orth and Gorbet 97] to demonstrate and verify the idea of Tangible Bits.
The surface of the metaDESK acts as a gateway between digital and physical space, allowing physical icons ["phicons"] to serve as containers and handles for physically manipulating digital information. The metaDESK also makes use of interface devices like the active lens, an arm-mounted LCD display, and the passive lens, a mobile "lens" of back-projected, fiber-cluster glass, as physical interface tools. The first prototype application of metaDESK is Tangible Geospace, which allows the user to interact physically with 2D and 3D views of the MIT campus [Fig. 4].AMBIENTROOM [ISHII AND ULLMER 97] The ambientROOM is a platform exploring the use of ambient media to communicate information at the periphery of human perception. Ambient displays such as light, shadow, sound, airflow, and water movement enable the room to communicate information about the world, such as weather, traffic, or the even the activities of your child.INTOUCH [BRAVE AND DAHLEY 97] inTouch explores new forms of interpersonal communication across distance that exploit touch. A prototype is made of two devices, each consisting of three cylindrical rollers embedded in a base [Fig. 5]. When a user rotates any or all of these rollers, the exact movement is mimicked by the partner device, even though it may be located far away. Users can thus feel the distant person's interaction through mutual manipulation of a "shared" physical object.TRIANGLES [ORTH AND GORBET 97] Triangles is a construction kit of identical, flat, plastic triangles that can be connected to each other both physically and digitally to trigger specific digital events, allowing a simple, but powerful means of physical interacting with digital information [Fig. 6]. The Triangles project is a first step towards a larger goal of developing 'dual-identity objects' – objects which, by their physical design, are capable of making the structure and relationships of digital information tangible.CONCLUSION Where the land meets the sea, countless creatures have evolved diverse and creative approaches for flourishing among the turbulence and constant change of these competing worlds. On the shoreline between the sea of bits and the firm ground of physicality, we face a similar challenge of reconciling our dual citizenship with the worlds of physicality and digitality, bits and atoms.
The land of atoms and sea of bits are currently worlds divided. The physical world lies sleeping, largely lacking in digital content. Our windows into cyberspace remain limited to "painted bits" leaking out of a myriad of rectangular screens, bound to limited interfaces which impoverish our senses, leaving unsatisfied our aesthetic sensibilities.
Herein, I have presented my visions of the ClearBoard and Tangible Bits, where the architectural surfaces, graspable objects, and ambient media which surround us may be transformed into seamless interfaces between bodies, bits, and atoms. I invite you to join me in returning to the seashore, and rejoicing in the sea, sand, and air, atoms and bits, joined together as one.
BRAVE, S. AND DAHLEY, A., MiLOs [Multi-Locational Objects], Ars Electronica Festival 97, Linz 1997
BUXTON, W., Living in Augmented Reality: Ubiquitous Media and Reactive Environments, Video-Mediated Communication, Lawrence Erlbaum Associates, 1997, pp. 363-384
HOLLAN, J. AND STORNETTA, S., Beyond Being There, in: Proceedings of CHI '92, ACM, New York, 1992
ISHII, H. AND ULLMER, B., Tangible Bits: Towards Seamless Interfaces between People, Bits and Atoms,in: Proceedings of Conference on Human Factors in Computing Systems [CHI '97], ACM, Atlanta, March 1997
ISHII, H., KOBAYASHI, M. AND ARITA, K., Iterative Design of Seamless Collaboration Media, in: Communications of the ACM [CACM], Special Issue on Internet Technology, ACM, Vol. 37, No. 8, August 1994
ISHII, H., KOBAYASHI, M. AND GRUDIN, J., Integration of Interpersonal Space and Shared Workspace: ClearBoard Design and Experiments, in: ACM Transactions on Information Systems [TOIS], ACM, Vol. 11, No. 4, October 1993, pp. 349-375
MANN, S. Smart Clothing: Wearable Multimedia Computing and "Personal Imaging" to Restore the Technological Balance between People and Their Environments, in: Proceedings of ACM MULTIMEDIA '96, November 1996, 163-174
ORTH, M. AND GORBET, M., Triangles and the Digital Veil, Ars Electronica Festival 97, Linz 1997
NORMAN TEN, Living on the Edge, in: Exploratorium Quarterly, Vol. 11, Issue 3, Fall 1987
WEISER, M., The Computer for the Twenty-First Century, in: Scienti½c American, September 1991, pp. 94-104
I thank Prof. William at the University of Toronto for countless discussions about skill-based design and foreground & background issues, through which many of the ideas in this paper were developed and shaped [Buxton 97]. Thanks are also due to Mark Weiser for his inspiring Ubiquitous Computing work [Weiser 91].
I thank TTT [Things That Think], a new consortium at the MIT Media Lab, for its ongoing support of the Tangible Bits project.
I also would like to acknowledge the contribution of many hardworking graduate and undergraduate students at MIT for work on the realization of Tangible Bits ideas. In particular, we thank graduate students Brygg Ullmer, Scott Brave, Andrew Dahley, and Matt Gorbet.
|
