|
Get in Touch
Tangible Bits: Bridging Digital and Physical
'Hiroshi Ishii
Hiroshi Ishii
At the seashore, between the land of atoms and the sea of bits, we are now facing the challenge of reconciling our dual citizenship in the physical and digital worlds. Our windows to the digital world have been confined to flat rectangular screens and pixels—or “painted bits”—while our visual senses are steeped in the sea of digital information, our bodies remain in the physical world. “Tangible Bits” give physical form to digital information, making bits directly manipulable and perceptible. Tangible bits pursues seamless coupling between these two very different worlds of bits and atoms, blurring the boundary between science/technology and arts/design.
BEYOND GRAPHICAL USER INTERFACES, TOWARDS TANGIBLE USER INTERFACES People have developed sophisticated skills for sensing and manipulating their physical environments. However, most of these skills are not used when we interact with the digital world. Interactions with digital information are now largely confined to Graphical User Interface (GUI) boxes. We are surrounded by a variety of ubiquitous GUI devices such as personal computers, handheld computers, and cellular phones.
GUI represents information (bits) with pixels on a bit-mapped display. Those graphical representations can be manipulated with generic remote controllers such as mice and keyboards. By decoupling representation from control in this way, GUIs let users control and represent a variety of media graphically. Using graphical representations and “see, point and click” interactions, the GUI is a significant improvement over its predecessor, the CUI (Character User Interface), which require users to “remember and type.” But interacting with pixels through GUIs is inconsistent with how we live. We cannot take advantage of our dexterity or use our skills for manipulating physical objects (e.g. building blocks) when we interact with the GUI world.
The Tangible User Interface (TU I) is significantly different from the GUI. The TUI gives physical form to digital information, serving as both a representation and a control. The TUI’s physical embodiment makes digital information directly manipulable with our hands and naturally perceptible through our peripheral senses.ABACUS—THE ORIGIN OF TANGIBLES The author met a highly successful computational device called the “abacus" when he was two years old. He could enjoy the touch and feel of the “digits” physically represented as arrays of beads. This simple abacus was not merely a digital computational device: because of its physical affordances, the abacus also became a musical instrument, an imaginary toy train, even a back-scratcher. He was captivated by the artifact’s sound and its tactile properties.
This childhood abacus was also a medium of awareness. When his mother kept household accounts, he was aware of her activities by the sound of the abacus, knowing that he could not ask her to play while the abacus made its music.
This abacus also revealed a new direction in Human-Computer Interaction (HCI) that we call Tangible User Interfaces (TU I). First, the abacus makes no distinction between “input” and “output.” Instead, its beads, rods, and frame all serve as physical representations of numerical information and computational mechanism. They serve as directly manipulable physical controls to compute on numbers. Second, the simple and transparent mechanical structure of the abacus (without any digital black boxes) provides rich physical affordances. Anyone can immediately understand what they can do with this artifact without reading a manual. The TUI design challenge is to seamlessly extend physical affordances into the digital domain.PROJECTS Pinwheels
James Patten, Gian Pangaro, Angela Chang, Sandia Ren, Phil Frei,
Rujira Hongladaromp, Craig Wisneski, Andrew Dahley
and Professor Hiroshi Ishii
Pinwheels is an ambient display that spins in a wind of digital information (bits). The spinning Pinwheels allow people to feel the flow of bits representing human activities or happenings in the natural world in their peripheral vision while they concentrate on other activities (such as conversation) in the foreground. An astronomer following the activities of a solar corona could install these Pinwheels in his or her home in order to monitor solar winds in the background. Being peripherally aware of subtle changes in solar activity leading up to significant events could help the astronomer time periods of intensive observation. The basic concept is to make solar winds of ionized particles and all kinds of other information flows perceptible in architectural space as a “wind” driving old-fashioned Pinwheels.
Current graphical user interfaces display most of the information as pixels on a screen, requiring the user’s conscious attention. As such they are foreground media. But our capacity to recognize and process information is exhausted when we are faced with too much data in the foreground, leading to information overload. Ambient displays, such as spinning Pinwheels, help to solve this problem by representing continuous information flows as continuous physical phenomena in the background so the user can be aware of them peripherally.
inTouch
Angela Chang, Matthew Malcolm, Phil Frei, Victor Su, Rujira Hongladaromp,
Scott Brave, Andrew Dahley, and Professor Hiroshi Ishii
inTouch is a project that explores new forms of interpersonal communication through touch. inTouch uses force-feedback technology to create the illusion that people—separated by distance—are actually interacting with shared physical objects (Distributed Shared Physical Objects). Each of two identical inTouch devices use three freely rotating rollers. Force-feedback technology synchronizes each individual roller to the corresponding roller on the distant mechanism; when one inTouch roller is moved the corresponding roller on the other inTouch also moves. If the movement of one roller is resisted, the corresponding roller also feels resistance. Two distant users can play through touch, moving rollers to feel the other’s presence. inTouch demonstrates a unique interface that has no boundary between “input” and “output” (the wooden rollers are force displays as well as input devices). The sense of touch is playing critical role, and information can be sent and received simultaneously through one’s hand. Past communication media (such as video telephony) tried to reproduce the voice or the image of the human face as realistically as possible in order to create the illusion of “being there.” inTouch takes the opposite approach by making users aware of the other person without explicitly embodying him or her. We think that inTouch creates a “ghostly presence.” By seeing and feeling an object move in a human fashion on its own, we imagine a ghostly body. The concept of the ghostly presence provides us with a different approach to the conventional notion of telepresence.
Pegblocks
Ben Piper, Matthew Karau, Beto Peliks,
and Professor Hiroshi Ishii
Pegblocks are networked tactile transducers. As users manipulate the array of pegs, sliding them back and forth, motion is converted to electricity and converted back into motion through out the rest of the network. The resulting movements of the pegs is not determined by any one individual input but by the networked group as a whole. Each peg is coupled to an electric dynamo/motor. The dynamo/motor can act in two modes: it can generate electricity from motion and convert electricity inton motion. As the peg is moved back and forth electric energy is generated and converted back into motion through out the rest of the network. Pegblocks extend the notion of Distributed Shared Physical Objects of inTouch to explore re-configurable, haptic communication network, integrating representations using peg patterns that were impossible in inTouch. They allow the simultaneous touch of multiple human hands to be extended over space, as well as asynchronous exchange of peg patterns. Pegblocks is based on an early prototype developed by Ben Piper as a class project in a Physics and Media class taught by Prof. Neil Gershenfeld, MIT Media Lab, in the fall of 1999.
Curlybot
Phil Frei, Victor Su, and Professor Hiroshi Ishii
curlybot is an educational toy that records and plays back physical motion. When the user takes hold of curlybot and moves it around on a flat surface it remembers how it has been moved. When it is then released, it replays the movement with all the intricacies of the original, including every pause, acceleration, and tremor of the user's hand. It was designed to help children develop geometrical thinking and as a medium for lyrical expression. Phil Frei created the curlybot concept in late 1998, completing the industrial and interaction design himself. With the support of Victor Su for electronic circuit design and prototype construction, the first prototype was completed in the spring of 1999. The forcedfeedback technology used for real-time simultaneous communication in inTouch is used in curlybot for the recording and playback of non-simultaneous gestures. This project has significance in terms of both interface design and the use of computers for educational pUrposes. As a tangible interface it blurs the boundary between input and output (similar to inTouch): curlybot itself is both an input device to record gestures and a physical display device to re-enact them. By allowing users to teach curlybot gestures hand and body motions, curlybot enables a strong connection between body and mind not obtainable from anything expressed on a computer screen. From an educational standpoint curlybot opened new horizons as a toy to aid in the acquisition of geometrical mathematical concepts.
Triangles
Matt Gorbet, Maggie Orth, Ali Mazalek, Emily Cooper,
James Hsiao, and Professor Hiroshi Ishii
Triangles, originally created by Matt Gorbet and Maggie Orth in 1996, is a construction kit that consists of flat Triangles, each of which has both a physical and a digital identity. The user can use the kit to simultaneously create physical arrangements (including both two-dimensional tiles and three-dimensional structures) and digital representations that correspond to the tangible structure. Each physical triangle has magnetic, electrically-conducting connectors on each side. When they connect with each other, a communication circuit is formed and each Triangle conveys its identity and relative position to the host computer. Changes in the Triangles’ physical configuration trigger pre-programmed digital events to enable direct manipulation of the topological structure of information. Creating new tangible interfaces by giving new digital functions to objects which are already constrained by specific familiar meanings (such as a pencil which we understand as a tool for writing) fundamentally limits our freedom to ascribe digital meaning. To overcome this limitation, Triangles p“Urposely” uses the triangle’s abstract form to let users express and manipulate information. While the pencil is a tangible tool for creating unconstrained representations in a single medium, Triangles is a unique authoring system that lets users create and manipulate multimedia structures through physical actions.
bottles
Ali Mazalek, Dan Overholt, Jay Lee, Joanna Berzowska,
Rich Fletcher, Seungho Choo, Joe Paradiso, Charlie Cano,
Colin Bulthaup, and Professor Hiroshi Ishii
Through the seamless extension of physical affordances and the metaphor of bottles, this project explores interface transparency. Just as we naturally open and close lids to access bottles’ physical contents, in this project users open and close lids to access digital information. A wide variety of contents (including music, weather reports, and stories) have been developed to test the concept. The “bottle-as-interface” concept began as a “weather forecast bottle,” which Ishii envisioned as a present for his mother. Upon opening the weather bottle, she would be greeted by the sound of singing birds if the next day’s weather was forecasted to be clear. On the other hand, the sound of rainfall would indicate impending rain. Such an interface would be consistent with her everyday interactions with her familiar, physical environment, such as opening a bottle of soy sauce. She had never clicked a mouse or typed a URL in her life, but had opened soy sauce bottles thousands of times. In late 1998, Ishii and Rich Fletcher expanded this idea into “musicBottles” and began the project. They used sensor technology developed by Dr. Joe Paradiso and collaborated with different designers, engineers and artists to create a custom table and bottles with special electromagnetic tags. Three sets of bottles—each with different content: classical, jazz, and techno music— were designed and built. In June 2000, this project received the IDEA 2000 Silver Prize (International 2000 Industrial Design Excellence Awards competition). We developed custom wireless sensing technology for this project. An antenna coil attached to the underside of the table creates a magnetic field above the table. A custom electronic circuit detects disturbances in this magnetic field that are caused by the placement and opening of tagged bottles. The system then executes musical programs for each bottle (e.g. opening one bottle plays a piano) and controls the patterns of colored LED light projected onto the table. This project uses a combination of artistic and technological techniques to support emotional interactions that are fundamentally different from conventional, function-centric interfaces.
PingPongPlus
Jay Lee, Matthew Malcolm, Blair Dunn, Rujira Hongladaromp,
Craig Wisneski, Julian Orbanes, Ben Chun and Professor Hiroshi Ishii
Ping-pong is essentially an aggressive and competitive game. By introducing a variety of digital interaction layers into the game, it can be transformed into an experience with an entirely different pUrpose. The value of PingPongPlus is to let users learn about – in a fun and engaging way – the “transformations” of meaning that arise from the design of interactive digital layers. The basic concept of PingPongPlus is to use a standard ping-pong paddle and ball to transform the sport of ping-pong itself into an interactive experience. In PingPongPlus, digital water covers the surface of the ping-pong table. Each time a ball bounces off the table, digital ripples spread out quietly and the fish scatter. The theme of the research is to learn by experience how the digital layer projected on top of the ping-pong table affects the competitive interaction of the game of ping-pong. Eight microphones mounted on the bottom of the ping-pong table detect the sound waves emitted when balls bounce off the table surface. The coordinates of each ball are calculated by comparing the timing through an electronic circuit. Using this coordinate data, a computer program calculates the ripple patterns and the movements of the schools of fish, both of which are rendered in video image and sound by a video projector mounted on the ceiling above the table. In addition there is a mode for painting with the ping-pong balls and one in which the net on the table serves as a piano keyboard for playing music.
Urp
John Underkoffler, Daniel Chak, James Patten, Gian Pangaro, Jason Alonso,
Gustavo Santos, Benjamin Fielding-Piper, and Professor Hiroshi Ishii
Urp is a tangible urban-planning workbench based on the “I/O Bulb” concept originally developed by Dr. John Underkoffler in 1998. The “I/O bulb” creates high resolution, bidirectional light flows. It collects photons from physical surfaces, and uses knowledge about a particular domain, such as urban planning, to interpret the light patterns. It then responds with digitally-controlled light output, which is projected back onto the physical space. In Urp, physical architectural models are placed on a table illuminated with “I/O bulbs” and shadows are cast according to a computer simulation. By adjusting the clock, it is possible to track the shadow movements and sun reflections. In addition, air currents around the buildings are rendered visible and a wind gauge can be used to measure the wind speed at any point. Using “I/O bulbs” to project real-time computer simulations onto physical models makes it possible to understand and directly manipulate digitally rendered urban spaces in a world that is contiguous with one’s own body. Instead of the original computer vision technology, this installation uses new magnetic field-sensing technology to track tagged objects such as building models. When designing tangible interfaces, it is important to consider which elements should be given physical form and which elements should be rendered as digital images. The key to a successful interface lies in hiding the boundary between the digital and physical worlds. The digital shadows (video projections) cast by the physical models in Urp represent one solution to this problem. If we were to replace all of the hundreds and thousands of light bulbs in an architectural space with I/O bulbs, what kind of interaction design would be possible? The I/O bulb, as the core concept in this project, has demonstrated the potential for new digital interactions that occur not only on the tabletop, but within architectural space itself.
ClearBoard
Minoru Kobayashi (NTT), and Professor Hiroshi Ishii
ClearBoard’s goal is the seamless integration of interpersonal space and shared workspace. The project supports face-to-face interaction between two interlocutors who are collaboratively sharing and manipulating information. To realize this goal, we came up with a simple concept: people speaking to each other through a glass wall while drawing on both sides of it. Clear- Board combines video-drawing technology that allows simultaneous drawing on a common whiteboard and a video conferencing function that allows face-to-face conversation. Minoru Kobayashi and Hiroshi Ishii collaborated in the design and testing of this system in 1991 when they were colleagues at NTT Human Interface Laboratories. ClearBoard consists of two networked terminals connected by a video channel. Each terminal consists of a large screen tilted at a 45-degree angle and covered with a half-silver mirror; a video projector behind the screen from which images are projected; and a camera mounted on the ceiling. When one user draws on the screen with a fluorescent pen, the drawing, the face and the upper body of that user is reflected in the half-silver mirror captured by a camera on the ceiling. When this image is rear-projected onto the other screen, it creates the illusion that the other person is on the other side drawing on the screen. ClearBoard supports both simple eye contact and “gaze awareness,” letting each user know exactly what part of the common drawing surface the other is looking at. Eye movements are linked to thought processes and play an important role in non-verbal communication. Earlier conferencing systems were unable to provide gaze awareness. By inserting a common drawing surface between two separate spaces, ClearBoard is able to achieve isotropy while expanding the space at hand.
Credits Hiroshi Ishii, Director of Tangible Media Group and Exhibit / Rob Jacob, Associate Director of Exhibit / Mike Ananny, Assistant Director of Exhibit / Jay Lee, Exhibit Advisor / Virginia Siu, Project Coordinator of Exhibit / Kenroy Cayetano, Technical Director of Exhibit / Matthew Malcolm, Assistant Technical Director of Exhibit
Researchers Mike Ananny, Joanna Berzowska, Scott Brave, Dan Chak, Angela Chang, Seungho Choo, Andrew Dahley, Benjamin Fielding-Piper, Rich Fletcher, Phil Frei, Matt Gorbet, Jay Lee, Ali Mazalek, Maggie Orth, Gian Pangaro, Joe Paradiso, James Patten, Carlo Ratti, Sandia Ren, Victor Su, Brygg Ullmer, John Underkoffler, Paul Yarin, Luke Yeung, Craig Wisneski
Collaborators Jason Alonso, James Kenji Alt, Colin Bulthaup, Charlie Cano, Ben Chun, Emily Cooper, Blair Dunn, Saro Getzoyan, Rujira Hongladaromp, James Hsiao, Zahra Kanji, Matthew Karau, Minoru Kobayashi (NTT), Tim Lu, Julian Oranes, Dan Overholt, Joseph Panganiban, Beto Peliks, Gustavo Santos, Yao Wang Organized by Ars Electronica Center Special Thanks: M IT Media Lab, Things That Think and Digital Life Consortia, Steelcase Inc., NTT Cyber Space Laboratories, NTT Corporation 04_Engineers 02.08.2001 12:08 Uhr Seite 260
|
