Recent years have been marked by very many dynamic technical and scientific developments, all of them with a high potential to change our world, the way we think, the way we work, the way we do business.

Digitalization is often used as the overall term for these trends, which go as far as robotics and autonomous mobility, Internet of Things and smart environments or AI, machine learning and digital assistants. The new Ars Electronica program Future in a Nutshell is a special and unique opportunity to get an introduction and overview of these game-changing new technologies—understandable to everybody yet presented by selected experts from these fields.

MO Sept. 11, 2017

Credits

An event by the Austrian Tourism Days in cooperation with Österreich Werbung, WKO und BMWFW.

The characters have been learned by artificial intelligence (AI) not for their meaning but for their shape and patterns. The AI has created and drawn lines that look like characters but do not have any meaning.

This work was publicized at the Aichi Triennale 2016 international art festival. It was implemented by collecting handwritten artist statements or descriptions of work by an extremely broad international group of ten participating artists. By learning handwriting with one writer in each language, the artificial intelligence collected information on the shapes of each character system, as well the idiosyncrasies of each writer. The lines generated are written as if they mean something important; they also look deceptive.

Credits

Machine Learning Programming: Hironori Sakamoto

Supported by Nihon Unisys, Ltd., Haps

Sponsored by Japan Media Arts Festival and Bunkacho – Agency for Cultural Affairs, Government of Japan

Handwriting provided by: Valsan Koorma Kolleri, Lai Chih-sheng, Gulnara Kasmalieva & Muratbek Djumaliev, Kio Griffith, Ali Cherri, Taloi Havini, Song Sanghee, Shreyas Karle, Kawayan De Guia, Uudam Tran Nguyen

Zo, the latest social chatbot from Microsoft, is part of the Xiaoice family, which has chatted with over 100 million unique users worldwide. Zo holds the record for Microsoft’s longest continuous chatbot conversation: 1,229 turns, lasting 9 hours and 53 minutes.

Credits

App Development : Mina Khan
Stage Design: Penny Webb
Product Design: Hongxin Zhang
Supported by Microsoft

www.zo.ai

Faculty: Christa Sommerer, Laurent Mignonneau, Masaki Fujihata, Michaela Ortner, Fabrizio Lamoncha

Students in the Interface Cultures master’s program come from diverse countries and cultures across Europe, North America, Asia, Australia, South America and the Middle East. Their previous education includes various fields such as fine arts, media arts, design and engineering. Once in Linz, students spend two to four years at the University of Art and Design and immerse themselves in an international study program that teaches interaction design, interface technologies and the cultural and artistic context of user participation.

Students experiment with the development of interactive prototypes, musical interfaces, conceptual installations and interaction critiques and also broaden their view by encountering and collaborating with other international fellows who are investigating these topics from a different cultural context.

All the artworks, prototypes and various interfaces in this year’s exhibition have been made in Linz. Linz is the UNESCO City of Media Arts, and of course the cultural and socio-political context of Linz also impacts on the students ideas and concepts. It is the cumulative know-how of 37 years of Ars Electronica and the strong history in the electronic arts that enables a profound discourse around media technology and media arts specific to Linz.

But what does Made in Linz actually mean? Is it a proof of quality? When we look at various trade labels such as Made in China or Made in Germany or Made in Italy, we immediately associate certain product qualities, such as low prices, high-precision technology or fashionable design. But what about Made in Linz? What do we expect when we hear this term? At Interface Cultures this year we are proposing this term and leave it up to the visitors of the exhibition to reflect upon this topic. Perhaps it is even possible to establish this new term as a symbol of innovation, analysis and creative investigation of art and technology in our society. We invite everyone to discuss what Made in Linz means in the context of media art.

Credits

Supported by the Federal Ministry of Science, Research and Economy under the Higher-Education Structural Fund Austria

The Sung Portrait

Alexandre Gomez (FR), Isadora Teles de Castro e Costa (FR)

Come and see what they can do, these little creatures who live in a small screen, waiting for a seductive singer. But be careful, if you scream they run away, and silence makes them sleep. If you sing a little song to make them dance, you will discover a beautiful surprise: a reflection of a thousand colors.

Spirit Spaces

Aesun Kim (KR), Stevie Jonathan Sutanto (ID)

The emergence of digital media has become the tool and study in the field of art. The wearable device is a new tool for works in many performances using these media. Spirit Spaces is the study of a gap between the digital medium and humans, and it proposes the possibility of a new expression that organically interacts through the wearable device. Aesun Kim and Stevie Jonathan Sutanto used a gravity physics model for the organic expression of body. Light can be an energy space to help share their personal air.

Polyus

Johannes Wernicke (AT)

Polyus is an omnidirectional speaker that allows multiple listeners to perceive different sounds from the same source simultaneously. Using Polyus, sounds can be positioned in specific areas of a room that will be only perceived by the listener in the area that a sound is assigned to. The system consists of three core components: the “Acouspade”, a directional speaker which can focus sound into a narrow beam, a reflector redirecting the sound while spinning at high velocity and a LIDAR (light detection and ranging) sensor tracking the visitor’s position. It allows the creation of nonlinear, spatial compositions through which the audience can move, rather than perceiving it on a timeline. The system is also intended to test our ability to orient ourselves using our hearing.

Neiema נְעִימָה

Or Wolff (IL)

Neiema (Hebrew)—melody, voice, tone. Implicit characterization.

This project was created to get an insight into the world of the artist and to explore how sound can reflect a story with varieties of interaction with different people. This installation presents a connection between visual graphics that transform into sound by the audience touching many kinds of pattern prints. These pattern were created using a microscope lens to produce uniquely detailed graphics. This kind of technique represents transparent looks inside the artist’s most private world. The action of touching allows a more exposed, sensitive and personal connection with the content of the project. Everyone creates different kinds of Neiema (melody), which change according to the way they interact with the graphics.

The Murder of Jo Cox

Supergraph: Thomas Hoch (AT) and Waiwai (HK)

Dazzled by the hyperreality of digital journalism we may find ourselves questioning the possibility of truth among contradictory narratives and sensations. British MP Jo Cox’s death in June 2016 has represented such a scenario most archetypally with an extraordinary display of media bias provided by the so-called left and right, from national newspapers to independent media.

In this work, the artists aim to create a user-experience of robotic journalism. Audiences can map out and generate news using artificial intelligence and data mined from a range of UK media online by using a mixer interface. The result will be displayed in an installation composed of data visualization, a ticker and research material.

mobMess

Mario Gomez (ES)

mobMess is a social interface project dedicated to revealing the capabilities of mobile technologies for participatory art. Through a simple interaction you can transform your smartphone screen into one pixel in order to synchronize with other participants and create a collective display. Gathering to render a message, showing synergy in a team effort and realizing the cohabitants of the same time and space, a mirror of crowd individuals emerges. The aim of developing this project is to have an interface from which a concept can transform and scale uncanny performances up or down. And further, to make a tool to revindicate the power that underlies technology, communication and art.

Make-A-Pick

Onur Olgaç (TR)

Make-A-Pick is a game of chance with binary selection that plays with the psychological concept of the gambler’s fallacy. It uses a simple form of play that is at the heart of Roger Caillois’ definition of Alea: “All games that are based on a decision independent of the player . . . in which winning is the result of fate rather than triumphing over an adversary. More properly, destiny is the sole artisan of victory, and where there is rivalry, what is meant is that the winner has been more favored by fortune than the loser.” The interactive installation acts as an interface for the visitors to take on the challenge of finishing the game by testing their luck. Presented with two choices to pick from, the visitors make a pick to further improve the current streak. If they make a wrong move, the whole progress is lost and they have to start over. The main question behind the game is asking the visitors whether they fall into the trap as a result of their own intuition or if they are able to figure out a pattern to beat fate and win.

Lost, but not lost forever

Monica Vlad (RO)

The concept of this project is to use old media devices to create a sound performance. Monica Vlad will use a sewing machine, two radios, one tape recorder, a cassette player/Walkman, a turntable and a vintage camera. All the devices are connected to the mixer. Contact microphones are used to take the sound from some devices and amplify it. Piezo elements take the sound from the vibrating surface of some other devices. The other devices are directly connected to the mixer. Vlad has also created a cassette tape loop that serves as an analog loop. Analog effects are applied to most of the devices. The “new” instruments are played sequentially to create the sound for the performance.

Leaves

Monica Vlad (RO)

Leaves is an installation that consists of real-time rendered graphics in a screen and timeline UI in smartphone displays. It is a monument like a data-visualization to think about mortality caused by mental illness, especially suicide. Every suicide is a tragedy that has long-lasting effects on the people left behind. But they can hardly share a grief that they are trying to hide.

This is a real-time simulation based on statistics for worldwide mortality details published by the WHO. This project aims to enable sensing, thinking about, talking about or just feeling each incidence of death.

Fuzzy_Logic Machine

Gabriela Gordillo (MX), Irene Ródenas Sáinz de Baranda (ES)

Fuzzy_Logic Machine is an instrument designed to control the values of a sound device through analog manipulation. A set of sensors are triggered by light parameters that are arranged in a grid (X,Y,Z position). Each value can be manually accessed, connecting image and sound through a spatial variable. Thus the mechanism reproduces the inner structure of the audible result while it triggers its changes in a feedback relation between the two mediums. The instrument becomes an interface in which performers and spectators share a similar perception about the variables that compose the sound. The motion of the lights guide the cognitive approach of the spectator’s listening experience, as well as the creative experience of the performer, revealing what is behind the interface.

The instrument questions the need for self-made systems that allow a different contact with technology through the understanding of its logic. By doing this, it aims to take complexity and express it through simplicity.

Echo Chamber

Luis Toledo (ES)

We are constantly being bombed by advertising, news and ideas. Inside the Internet domain, we are able to mute uninteresting topics or even subscribe to new sources. To achieve profit, social media shapes our profiles and distributes content that fits our preferences.

During the exhibition, a database is fed by the active participation of users in the room, who vote real-time news with a thumbs up or thumbs down. Up-voted articles and keywords are used by a Twitter bot to build an almost random opinion, posting a comment every 30 minutes.

By simplifying the internal mechanisms of today’s social networks to customize and deliver information, Echo Chamber highlights topics such as information diversity, manipulation and opinion realms, and how our current use of technology is letting us hear only what we want to hear.

Communication Noise

Julia del Río (ES)

Julia del Río explores diverse artistic strategies for interaction within electromagnetic fields, especially in sound performances. Her concerts sonically translate the invisible world of interference and magnetism without musical compromises. Her sound is always a result of an interchange and of various acts of digital communication. She presents Communication Noise, a participatory audiovisual performance where the artist sonifies the electromagnetic waste produced by cellphone interaction.

Bitcoin Traces

Martin Nadal (ES)

It is suggested that the future of money is crypto currencies and the most relevant of these is bitcoin. The main difference between fiat money and bitcoin is that money is not created by government regulation or law, but generated by a competitive and decentralized process called mining, and that all transactions are stored publicly in a common ledger called the blockchain.

Bitcoin Traces draws an infographic data-visualization of a transaction from the moment the bitcoins involved were created by a miner. Not focusing on the role of money as a measure of value but rather exploring its history, in which other transactions have participated in the past, depriving the money of materiality. Considering money as a network where each node is a good or a service and each edge a transaction it participates in.

Uterus

Klimentina Hristova (BG)

Uterus is a three-dimensional light and sound installation, providing the opportunity for sensorial experiences of one’s own emotional habitat. Going beyond concepts of emotionality as “something inner” expressed by “something outer,” encoding and decoding are parts of the emotional process themselves. This raises the question of emotional articulations, the relation between cognition and sensory-motor aspects of habitualized emotions, and their relationship to the imaginary as well as proprioception. Exploring the relationship between body conditions and feelings, considering the simultaneous character of our mental health and being, we reach the idea of safeness. Like the child’s instinct to find a place to hide out, in our grown-up version we should find these in-between points where we can express ourselves freely.

The device transmits the touch sensations of kissing by measuring the lip pressure of the users and replicating this pressure through the movements of linear actuators. It has a lip-like sensing interface made of a soft and flexible rubber material that the user interacts with. An array of force sensors and linear actuators measures and generates real-time force feedback at various points on the user’s lips. The device is connected to a mobile phone, so that you can have a video call with your loved ones while using the device to send them a kiss.

With Kissenger, people can communicate deep emotions, and maintain physical intimacy and close relationships from any part of the world through the Internet.

Credits

City University of London and Imagineering Institute

www.adriancheok.info
www.emmayannzhang.com

www.kissenger.info

AI has made sure there always is! A “speech race” is upon us. First we had interactive voice response systems, now with natural language interface systems we have our new “weavers of speech,” these modern day “voices with a smile” are changing the way we communicate with our phones. Siri, Alexa, Bibxy, Cortana and Google Assistant (shall we call her GAbby?) are all vying for your attention, but what will our budding relationships with these Boy/Girl Fridays blossom into? Hello Machine—Hello Human, touches on the playful moments that are shared between man and machine, and seeks to connect with you by inverting this relationship, by asking what can you do for her.

Hello Machines are situated across the globe in ever-changing locations and time zones. Picking up the receiver rings the other Hello Machines, creating space for spontaneous voice visiting. They provide a way in which the viewer can interact with re-animated, technically obsolete telephone systems, utilizing present-day advances in telephony. Their aim is to open up a dialog between the technologies’ original ideas and meanings, and what makes up the “thingness” these devices now possess, by unraveling its historical and societal content that contains traces of our identity.

Credits

This project has been assisted by the Australian Government through the Australia Council, its arts funding and advisory body.

Hello Machine—Hello Human was developed within the Ars Electronica Futurelab, and forms part of Rachel Hanlon’s PhD Research through Deakin University, Australia.

The assistive interfaces pursue the development of devices to enable people with physical and mental disabilities and/or immobility to talk, produce and teach arts—including three-dimensional scenery and sound. The theoretical basis is in studies on multimodality and multi-sensoriality originating from neuroscience researches.

All prototypes are developed from re-engineering, customization, low-cost technologies and open-source codes. The work of the scientists does not need high technology; on the contrary, they propose using old knowledge available to non-experts in order to allow broad access to create all kinds of art in a collaborative way.

Credits

Special thanks to Juntos Com Você (Together With You), a foundation for social crowd-funding who made it possible to collect the funds needed for this exhibition and to all donors who made this campaign a success. Thanks also to the PPG in Arts of the Institute of Arts of UNESP, the FAPESP, CAPES and the CNPq for the research grants provided to the members of the GIIP. Thanks are also due to FAPESP for their aid to be part of the trip for this event.

It should be emphasized that without the research by Efraín Foglia, Ana Amália Barbosa, Renato Hildebrand and Daniel Paz, in the first phase of the assistive interfaces project, as well as Samara Andressa Del Monte and her family, it would not have been possible to develop and analyze the current state of the research.

Research Groups:

GIIP—International and Interinstitutional Group for Research on Convergences between Art, Science and Technology, Arts Institute, State University of São Paulo (UNESP)

Director: Rosangella Leote

cAt—Research Group Science / Arts / Technology, Arts Institute, State University of São Paulo (UNESP)

Directors: Milton Sogabe and Fernando Fogliano

Realidades—Research Group Realities: from tangible realities to ontological realities, School of Communications and Arts (ECA), São Paulo University (USP)

Director: Silvia Laurentiz

Design and Body Research Group, University Anhembi/Morumbi

Director: Agda Carvalho

Não se aproxime / Do not come closer

Dr. Rosangella Leote (BR), Fernanda Duarte (BR), Rodrigo Rezende (BR)

The handcuffed body of the performer is video projected, merging the naked form and the clothed forms and inviting the visitor’s approach. When approached, ultrasound sensors feedback the signal and the interactor modifies the image. The closer the visitor gets to the screen, the more the naked version of the performer blurs and disappears.

Collaborator: Miguel Alonso (BR)

Hatred Apparatus

Fabrízio Poltronieri (BR), Nicolau Centola (BR), German Alfonso Nunez (BR)

This apparatus connects to the web and automatically captures the comments of news-website users. Stored on a database, these are automatically posted on Facebook and Twitter in a randomized way. The intention of the play is not to condemn the comments themselves but instead to show them to the public in a setting detached from their original and usual context—the daily social-net stories.

RONIN—Wearable object

Agda Carvalho (BR), Edilson Ferri (BR)

This work of art proposes a reflection about human displacement (nomadic state) caused by conflicts and pressures inflicted to a group or individual, which are characterized by an apparent sense of disorientation in routes and movements. RONIN is a wearable object similar to an armour that shows information about the variation on these routes and movements on the surface of the clothing.

Collaborators: Ariadne Cordeiro, Lisete Carvalho, Wellington Moreira, Miguel Jacobut

Sopro / The blow

Milton Sogabe (BR), Fernando Fogliano (BR), Fabio Oliveira Nunes (BR), Soraya Braz (BR), Carolina Peres (BR), Cleber Gazana (BR), Rodrigo Dorta Marques (BR), Mirian Steinberg (BR), Daniel Malva (BR)

Sopro is energized by the audience through the force of them blowing into a propeller, which creates electrical energy that is again turned into the movement of tiny motors on water. The artwork is based on the use of a simple technological system: the poetics of the act of blowing and the use of primary scientific principles. This system in progress is conceived parallel to energy and sustainability issues, placing them in post-digital thinking.

Quem Sou Eu Se Não Você Em Mim / Who Am I If Not You On Me

Daniel Malva (BR)

This work consists of GIF sequences generated using machine-learning techniques, the results of a Google tool called Deep Style. The images used in this series were captured on a smartphone; later the software was applied 40 times to a self-portrait to create modifications of the original image, representing the traces left by these people within the artist.

Enigma 3.2—φ: Um Enigma para Gibson

Silvia Laurentiz (BR), Marcus Bastos (BR), Cássia Aranha (BR), Dario Vargas (CO), Lali Krotoszynski (BR), Loren Bergantini (BR), Sergio Venâncio (BR)

Phi is part of a series of interactive installations called Enigmas, whose poetic audiovisual systems perform aesthetic operations reflecting the issues raised by authors such as Flusser, Bergson and Gibson. The artwork uses webcams for a real-time sensing of the variance and invariance of light in the exhibition area. All the data collected through this process is translated into monochromatic lines and synthesized sounds.

Driftscope

Objective Passional Terrains

Lucas Pretti (BR), Tiago F. Pimentel (BR)

Driftscope is a helmet that feels the city. Equipped with a Raspberry Pi 3, two cameras (objective and subjective) and five environmental sensors, the wearable apparatus is triggered by the artist’s facial expression while walking around. The collected data accumulates a city “psychotopology”. The experience is represented in an interactive installation.

http://terrenos-apaixonantemente-objetivos.cc/

Filmattraktionen 1:04

Jorge Ribail (CU), Rodrigo Rezende (BR), Fabio Rodrigues (BR), Fernanda Duarte (BR)

The work is an interactive video installation in which a software edits and presents an approximately one-minute cinematographic sequence composed from a political quiz and answered by the interactor. The ultimate goal of this installation is for the organization of cinematographic discourse to equate to the processes that take place in the human cognitive system.

Immersive.architecture.generator_v17.07.03

Alexander Peterhaensel (DE)

The installation shows excerpts from a recording of a live interaction with the immersive architecture generator. It documents the user interaction with an audiovisual virtual environment which is playable like a musical instrument. The work is a virtual-reality application that allows the user to interact with an adaptive virtual architecture which responds to their behavior in real time.

Somewhat like being alive (Reality collider)

Rodrigo Rezende (BR), Lucas Gorzynski (BR), Fernanda Duarte (BR), Helimar Macedo (BR)

Somewhat like being alive (Reality Collider) is an interactive audiovisual work where the public has a small device (a gyroscope) generating data by its rotation and displacement. This data determines the images to be projected on the wall, as well as several of its properties and the sounds to be reproduced by amplifiers.

Sketch of Making art with the eyes spectacle

Dr. Rosangella Leote (BR), Nigel Anderson (BR), Daniel Paz (BR)

The pocket performance consists of the interactions between an assistive technology interface (an eye-tracker device developed in GIIP: the ARTIA.V) that is used by the public, a dancer with trackable tags on the body and live sound composition. The work will show part of the dance spectacle Making art with eyes presented in Brazil (2017).

Teclaut

Dr. Rosangella Leote (BR)

Teclaut is a fully adaptable analog input interface that can be used by people with a range of disabilities – especially in cases of cerebral paralysis and locked in syndrome – as long as any part of the body can be moved to interact with the device. Its social potential is vast. Teclaut is highly customizable and requires about an hour to set up. It can also be used for a class activities in schools or as support for literacy.

ARTIA Project

Dr. Rosangella Leote (BR), Rodrigo Rezende (BR), Rodrigo Dorta (BR) and Daniel Paz (BR)

ARTIA.V is an interface that uses eye tracking to produce and teach arts. In the future the following areas will be supported: 2D and 3D drawing and painting, sculpting via 3D printing, photography and communicating in writing, and speaking by sound bank or via reading / automatic text recognition.