Kobito – Virtual Brownies


'Takafumi Aoki Takafumi Aoki

One common way to create imaginary, virtual creatures is to overlay computer graphics images on real scenes. But this method is not sufficient, because it allows people to only see the imaginary creatures. In Kobito-Virtual Brownies, imaginary creatures interact with the real world. They move real objects, and people interact with them through the real objects. The real objects function as a kind of “Haptic interface.”

The goal of this project is to make people feel Kobitos as “creatures that exist in our real world.” To make Kobitos exist virtually and naturally, it is necessary to avoid using markers, head-mounted displays, or wearable sensors, which are very common in virtual reality systems, but unfamiliar in daily life. And users must be able to interact with Kobitos in various ways.

The interaction between the Kobitos and a tea caddy is realized with physical simulation. Usually, physical simulation deals only with imaginary objects, but in this case, it includes a real object (the caddy). This is realized by synchronizing the movement between the real caddy (in the real world) and the imaginary caddy (in the physical-simulation world). Attendees can see the interaction through the Kobito window, which displays a combined image of the Kobito CG and the image of the real scene. No previous virtual reality system has enabled imaginary creatures to move real objects.

Kobitos can be good playmates, because they are sometimes invisible and at other times visible. When they are invisible and they move something in the real world, they generate a sense of wonder. Interaction with such a creature could be a new type of entertainment. Beyond entertainment, this technology could be very useful in daily life. Kobitos are intermediates between physical agents and imaginary agents, so they could act as agents between human beings and machines.

Kobitos are a new type of agent. Usually Kobitos don’t bother you. Whenever you want to see them, you can see them through the Kobito window. The most important thing is that Kobitos work even when they are invisible.

Takafumi Aoki, Hironori Mitake, Rikiya Ayukawa, Takatsugu Kuriyama, Toshihiro Kawase, Takashi Matsushita, Takashi Toyama, Hiroshi Ichikawa, Kazuyuki Asano, Itaru Matsumura, Yuichiro Iio, Makoto Sato, Shoichi Hasegawa, Society for the Study of Robotics; http://rogiken.org; Supported by Tokyo Institute of Technology