The assistive interfaces pursue the development of devices to enable people with physical and mental disabilities and/or immobility to talk, produce and teach arts—including three-dimensional scenery and sound. The theoretical basis is in studies on multimodality and multi-sensoriality originating from neuroscience researches.

All prototypes are developed from re-engineering, customization, low-cost technologies and open-source codes. The work of the scientists does not need high technology; on the contrary, they propose using old knowledge available to non-experts in order to allow broad access to create all kinds of art in a collaborative way.

Credits

Special thanks to Juntos Com Você (Together With You), a foundation for social crowd-funding who made it possible to collect the funds needed for this exhibition and to all donors who made this campaign a success. Thanks also to the PPG in Arts of the Institute of Arts of UNESP, the FAPESP, CAPES and the CNPq for the research grants provided to the members of the GIIP. Thanks are also due to FAPESP for their aid to be part of the trip for this event.

It should be emphasized that without the research by Efraín Foglia, Ana Amália Barbosa, Renato Hildebrand and Daniel Paz, in the first phase of the assistive interfaces project, as well as Samara Andressa Del Monte and her family, it would not have been possible to develop and analyze the current state of the research.

Research Groups:

GIIP—International and Interinstitutional Group for Research on Convergences between Art, Science and Technology, Arts Institute, State University of São Paulo (UNESP)

Director: Rosangella Leote

cAt—Research Group Science / Arts / Technology, Arts Institute, State University of São Paulo (UNESP)

Directors: Milton Sogabe and Fernando Fogliano

Realidades—Research Group Realities: from tangible realities to ontological realities, School of Communications and Arts (ECA), São Paulo University (USP)

Director: Silvia Laurentiz

Design and Body Research Group, University Anhembi/Morumbi

Director: Agda Carvalho

Não se aproxime / Do not come closer

Dr. Rosangella Leote (BR), Fernanda Duarte (BR), Rodrigo Rezende (BR)

The handcuffed body of the performer is video projected, merging the naked form and the clothed forms and inviting the visitor’s approach. When approached, ultrasound sensors feedback the signal and the interactor modifies the image. The closer the visitor gets to the screen, the more the naked version of the performer blurs and disappears.

Collaborator: Miguel Alonso (BR)

Hatred Apparatus

Fabrízio Poltronieri (BR), Nicolau Centola (BR), German Alfonso Nunez (BR)

This apparatus connects to the web and automatically captures the comments of news-website users. Stored on a database, these are automatically posted on Facebook and Twitter in a randomized way. The intention of the play is not to condemn the comments themselves but instead to show them to the public in a setting detached from their original and usual context—the daily social-net stories.

RONIN—Wearable object

Agda Carvalho (BR), Edilson Ferri (BR)

This work of art proposes a reflection about human displacement (nomadic state) caused by conflicts and pressures inflicted to a group or individual, which are characterized by an apparent sense of disorientation in routes and movements. RONIN is a wearable object similar to an armour that shows information about the variation on these routes and movements on the surface of the clothing.

Collaborators: Ariadne Cordeiro, Lisete Carvalho, Wellington Moreira, Miguel Jacobut

Sopro / The blow

Milton Sogabe (BR), Fernando Fogliano (BR), Fabio Oliveira Nunes (BR), Soraya Braz (BR), Carolina Peres (BR), Cleber Gazana (BR), Rodrigo Dorta Marques (BR), Mirian Steinberg (BR), Daniel Malva (BR)

Sopro is energized by the audience through the force of them blowing into a propeller, which creates electrical energy that is again turned into the movement of tiny motors on water. The artwork is based on the use of a simple technological system: the poetics of the act of blowing and the use of primary scientific principles. This system in progress is conceived parallel to energy and sustainability issues, placing them in post-digital thinking.

Quem Sou Eu Se Não Você Em Mim / Who Am I If Not You On Me

Daniel Malva (BR)

This work consists of GIF sequences generated using machine-learning techniques, the results of a Google tool called Deep Style. The images used in this series were captured on a smartphone; later the software was applied 40 times to a self-portrait to create modifications of the original image, representing the traces left by these people within the artist.

Enigma 3.2—φ: Um Enigma para Gibson

Silvia Laurentiz (BR), Marcus Bastos (BR), Cássia Aranha (BR), Dario Vargas (CO), Lali Krotoszynski (BR), Loren Bergantini (BR), Sergio Venâncio (BR)

Phi is part of a series of interactive installations called Enigmas, whose poetic audiovisual systems perform aesthetic operations reflecting the issues raised by authors such as Flusser, Bergson and Gibson. The artwork uses webcams for a real-time sensing of the variance and invariance of light in the exhibition area. All the data collected through this process is translated into monochromatic lines and synthesized sounds.

Driftscope

Objective Passional Terrains

Lucas Pretti (BR), Tiago F. Pimentel (BR)

Driftscope is a helmet that feels the city. Equipped with a Raspberry Pi 3, two cameras (objective and subjective) and five environmental sensors, the wearable apparatus is triggered by the artist’s facial expression while walking around. The collected data accumulates a city “psychotopology”. The experience is represented in an interactive installation.

http://terrenos-apaixonantemente-objetivos.cc/

Filmattraktionen 1:04

Jorge Ribail (CU), Rodrigo Rezende (BR), Fabio Rodrigues (BR), Fernanda Duarte (BR)

The work is an interactive video installation in which a software edits and presents an approximately one-minute cinematographic sequence composed from a political quiz and answered by the interactor. The ultimate goal of this installation is for the organization of cinematographic discourse to equate to the processes that take place in the human cognitive system.

Immersive.architecture.generator_v17.07.03

Alexander Peterhaensel (DE)

The installation shows excerpts from a recording of a live interaction with the immersive architecture generator. It documents the user interaction with an audiovisual virtual environment which is playable like a musical instrument. The work is a virtual-reality application that allows the user to interact with an adaptive virtual architecture which responds to their behavior in real time.

Somewhat like being alive (Reality collider)

Rodrigo Rezende (BR), Lucas Gorzynski (BR), Fernanda Duarte (BR), Helimar Macedo (BR)

Somewhat like being alive (Reality Collider) is an interactive audiovisual work where the public has a small device (a gyroscope) generating data by its rotation and displacement. This data determines the images to be projected on the wall, as well as several of its properties and the sounds to be reproduced by amplifiers.

Sketch of Making art with the eyes spectacle

Dr. Rosangella Leote (BR), Nigel Anderson (BR), Daniel Paz (BR)

The pocket performance consists of the interactions between an assistive technology interface (an eye-tracker device developed in GIIP: the ARTIA.V) that is used by the public, a dancer with trackable tags on the body and live sound composition. The work will show part of the dance spectacle Making art with eyes presented in Brazil (2017).

Teclaut

Dr. Rosangella Leote (BR)

Teclaut is a fully adaptable analog input interface that can be used by people with a range of disabilities – especially in cases of cerebral paralysis and locked in syndrome – as long as any part of the body can be moved to interact with the device. Its social potential is vast. Teclaut is highly customizable and requires about an hour to set up. It can also be used for a class activities in schools or as support for literacy.

ARTIA Project

Dr. Rosangella Leote (BR), Rodrigo Rezende (BR), Rodrigo Dorta (BR) and Daniel Paz (BR)

ARTIA.V is an interface that uses eye tracking to produce and teach arts. In the future the following areas will be supported: 2D and 3D drawing and painting, sculpting via 3D printing, photography and communicating in writing, and speaking by sound bank or via reading / automatic text recognition.