The „Body-Brush“ project is a genuine merger of art concept and the technology of computer vision. It creates a human-computer interface that transforms human body gesture into 3D paintings in real-time. This work is inspired by the action painting of Jackson Pollock. The interface consists of a novel body motion capturing and analyzing system and output and a virtual canvas space that is filled with virtual 3D color-paint-brushstrokes as a result of the painter s motion within the 3D canvas.

Project Goals

To build a human-computer interface that can liberate human creativity and satisfy aesthetic needs: The interface will enable users to interact intuitively with the machine and no prior training is required to use it. The user s unique body language will be transformed into painting brushstrokes in the 3-D canvas space, which simulates the visual forms and architectural spaces of designated complexities. It will provide an aesthetic experience that is unprecedented.

To develop an effective real-time motion capturing and analyzing system to recognize the 3D body motion: This project aims at developing a motion sensing and analysis system which is inexpensive, portable, but effective to capture and analyze the body motion data in the x-y-z-dimensions in real-time. The system will not require the user to wear any specific color clothing or sensor devices. This will enable a more immersed interaction between the human and the machine.

The virtual 3D canvas of „Body Brush“ consists of infrared sensitive cameras and arrays of infrared lamps, which allow the dynamic movement of the painter to be captured within the virtual 3D canvas space. A piece of advanced computer vision software developed at City University analyzes the movement and body gestures of the painter and generates a mixture of color strokes and patches that float in 3D space in synchronization of the painter s movement. Color saturation, value and the density of the brushstrokes can be controlled through the painter s body gestures. In addition, an innovative feature of the Body-Brush called the Virtual Path simulates the effects of throwing paint on the 3D canvas. This is done by analyzing the acceleration of body movement.