The concept was developed out of three different areas: the application of sound in medical science, coronary artery bypass surgery, and machine learning. The performance is based on the assumption that in the near future a surgeon will work with an advising machine while in surgery.

The installation is built using the performer’s real heart from MRI scans, enlarging its actual size. It is designed to interact when the performer plugs in audio cables and bridges connections, as is the case in coronary-artery bypass surgery. During the performance, the storyline is led by the sound, the mixed video of medical images and the live performance from the webcam. The video and the sound not only lead the storyline but also present the machine, which gives instruction to the performer as a physician. The patient (the heart) being operated on symbolizes human consciousness and faith. The performance poses the question: If machines can reason even better than humans, will we as humans lose some abilities and not even believe ourselves anymore?

Credits

This project was realized in cooperation with Fraunhofer MEVIS and Ars Electronica Futurelab (Peter Freudling, Erwin Reitböck).

Trained on many hours of his weekly address footage, a recurrent neural network learns the mapping from raw audio features to mouth shapes. Given the mouth shape at each time instant, the artists synthesize high-quality mouth texture, and composite it with proper 3D pose matching to change what he appears to be saying in a target video to match the input audio track. This approach produces impressive photorealistic results.

Credits

GRAIL Lab @ University of Washington

The interactive elements enable visitors to use eye control to navigate through the museum and access additional information about the works of art they are looking at. High-resolution 360-degree video sequences are combined with a 3D audio narration, which provides an immersive experience in text and image as well as with spatially variable sound. Users can select from among three narrative strands. They can thus determine the storyline themselves as they move about among the works of art. A complete round trip through all the museum’s galleries and collections of antiquities as well as an exclusive behind-the-scenes look at the conservators’ workshop—Österreich Werbung’s VR production delivers total immersion in the collection of Vienna’s Kunsthistorisches Museum.

Here, you can be the director, operate the camera, or simply hang around and see how a documentary takes shape. Questions are welcome!

Macro videos of scars varying in sizes and severity are taken from their hidden spots and projected in large format. Tracked by a camera system, visitors can use their hands to influence the footage shown: by placing your hand anywhere on your body, footage from that area is shown. This relationship between the feel of a physical touch and what is seen on the screen creates an intimate experience for the viewer.

All of Us is an ongoing project, anyone interested can contact allofus@marlenereischl.com to make their scars part of the installation.

Credits

www.marlenereischl.com

In general, with their sound, the Etudes drag you inside the code, giving you the feeling of being lost between the numbers. That feeling grows when you experience the sounds being transcribed into the binary video projections, the unlimited uncountable pattern of blacks and whites.

Credits

SpectroDuo is Martyna Kosecka (PL) and Idin Samimi Mofakham (IR)