Ars Electronica 1992
Festival-Program 1992
Back to:
Festival 1979-2007


Talk with Gerhard Roth

'Florian Rötzer Florian Rötzer

Q.: In recent years the biology of cognition has become a new epistemological statement, above all due to the theories of Maturana and Varela. Here, it is particularly emphasized that our brain is an autopoietic system which is excluded from its environment i.e. that is only stimulated by perturbations to perform certain cognitive feats, which are characterized by self-referentiality. Is this theory valid from the perspective of brain research and where does the evidence for this lie?
A: The concept of exclusion, as developed by Maturana and Varela, has led to a number of misunderstandings. One would have to invest a lot of work in this in order to clarify it. First of all, this concept is contra intuitive. The problem is that an animal or a person must orientate itself with the environment, by means of its sensory organs. The brain is the organ which processes this sensory information, ultimately producing a conduct by means of which the animal or the person can survive in its environment. How at all could living beings successfully live in an environment if the brain was to be excluded from it? Maturana and Varela have not supplied an adequate answer to this question in their theory. This gap has also not really been bridged by the constructivists. For example, it is the task of cognitive brain research to find out in what sense the brain is excluded or not excluded.
Q.: You wouldn't therefore readily agree with the constructivist thesis that we construe our reality; that it is only an image which is not in contact with what is outside?
A: The answer to this is complicated. There is an exclusion of the brain in the sense that everything we perceive and experience is the result of the activity of our brain i.e. what is "outside" and what is "inside" for us is produced by the brain. In this respect there is nothing that comes in from "outside". This is a trivial part of the concept of exclusion. However, what is not trivial is the question as to how this construed world arises in the brain and how the organism can orientate itself at the same time with the environment. How can these paradoxes be resolved? The solution is that the brain naturally orientates itself with the environment with the aid of the sensory organs, by receiving signals from it. However, what the brain does on the strength of these signals is in no way determined by the environment. The brain of a person and an animal must always interpret the signals which come from outside and which, as such, are free from significance. This is where the only meaningful substance to "exclusion" can be found.
Q.: For sometime it was assumed that the brain was to be understood as an information processing system analogous to the sequential and hierarchically structured Neumann computer. At present, the model of the so-called neuronal parallel processing networks tends to thrust itself into the foreground as a model which is more efficient. Does this mean that what can be installed in the computer, in accordance with the model of the brain, really compares to what takes place in it?
A.: Firstly, not at all. The step from the sequential to the parallel processing computer is a purely technical aspect. It has been seen that a number of technical cognitive tasks such as image detection work cannot be successively worked out as this would take too much time. This, however, has really nothing to do with the actual problem of the brain, which is, in fact, to find out which environmental signals are of significance for itself. As to whether this is processed sequentially or in parallel does not play a decisive role for the problem of the brain as a semantic machine or a semantic system.
Q: Until now computer simulations of intelligent work have obviously not been able reach this area of semantics. Would this be a consequence of your statements that this cannot even be achieved?
A.: Computers that are built by us should do something that is of benefit to us. This may be simple or complicated, but the significance of what they do is stipulated by us. For the time being, we do not want computers that do something that they, themselves, want.
Q: But this would be conceivable?
A.: If we assume we could design computers which are built like us, that they could attribute significance to what they experience in accordance with internal rules, then, under certain circumstances they would do things that we did not want or that were irrelevant for us. For this reason it is of no interest to us to build semantic computers. Whereas, the brain must produce a behaviour that is of significance for itself and the organism it is located in, and not for an observer. The brain construes itself together with the organism and consequently, its rules, too, according to which it meaningfully perceives and acts. As long as we do not want such computers there will not be the problem of producing meaning in computers. It may well be very difficult to build such computers but it is not impossible if we could find out how natural cognitive systems attribute certain meanings to signals.
Q.: Would then the association of brain and body not also be decisive, as our body does not only determine our position in, and our interaction with the world, but also evokes our needs and wishes? Would artificial intelligence then have to be implemented in a robot if it were to achieve the dimensions of semantics?
A.: One cannot understand the brain if one does not understand the senso-motoric environment it exists in. We know from people and from many animals that they do not construct a cognitive world if they are not active in the world. We know from babies and small children that they must actively experience and comprehend the world so that what they experience as the perceptive, imaginative or intellectual world, can develop. One result of the investigations of these relationships between brain research and artificial intelligence is that one would have to build computers as senso-motorically acting systems if they were really to be intelligent. They must be able to make tangible experiences with the world in order to develop an internal cognitive world. As to whether, as said, we want that and as to whether this can be realized technically is another matter.
Q.: The analogy of 'computer and brain' is supported by the fact that neuronal language is comparable to digital code and is not specific. It translates all the signals into a yes-no code, only transmitting intensities but not the qualities of the stimuli received by the individual sensors. What is obvious is merely the locality where stimuli are received and processed in the brain but their quality is not decisive as to how it is interpreted. Is it really correct that only the locality should decide what we perceive to be a visual, auditive or tactile impression? Similar to the way we can transform every bit sequence into any output by means of the corresponding peripheral equipment?
A.: Basically there is the principle of uncertainty or neutrality of the neuronal code i.e. the activity of the nerve cells has primarily nothing to do with what we perceive subjectively when the nerve cells are active. When a nerve cell is active, this can be in the context of seeing, hearing, smelling, in the context of colour or shape or movement, but one cannot read this from their activity. Now, there are of course, different types of coding on which what we ultimately perceive is founded. What nerve cells "can do" is be either active or not active, be either inhibited or stimulated and take in quite different stages of stimulation. Only quantative differences can be expressed with these different stimulation stages, e.g. the degree of brightness of a colour, the volume of a sound or the speed of a stimulation. Everything that is qualitative is coded according to other principles. We know that modality, that is the elementary difference between seeing, hearing, feeling, tasting, etc., takes place according to the locality principle, i.e. the modality is determined by where a stimulation takes place in the brain. However, where the stimulation really comes from is quite irrelevant. Whether I artificially stimulate a region of the brain like the occipital cortex, or whether the stimulation originates from the eye, what arises is a visual impression. Here, the neuronal code is a spatial code. This spatial coding is also valid for the so-called primary and secondary qualities, e.g. that a visual impression is perceived as colour. Then, of course, there are a number of very complicated codes which say something about the newness, the familiarity or the sense of stimulations. This is based on comparisons of comparisons between neuronal stimulations. The stimulation of nerve cells is always compared with the stimulation of other nerve cells. Meaning always arises in the brain proportionally. From this, the brain construes i.e. the parts of the brain construe the world together. There are certain spatial and temporal principles according to which the brain obviously strictly proceeds. When something happens at a certain time, at place A, then this is seeing, for example. If this happens at place A and something happens at place B this is, for example, either familiar or unfamiliar, meaningful or not meaningful.
Q.: Stimulations come from our sensory organs then, which translate into neural language and which are then computed into certain phenomena of seeing or hearing, to a certain extent. Would the metaphor then be correct, that our perceptive world is projected onto a mental screen? We do not see towards the outside, but onto a screen on which the outside is simulated.
A.: No, this metaphor would not be correct as we would then need someone to look at this screen. This, by the way, is one of the central problems of brain and cognition research. One discovers that the different codings that I have just described, take place simultaneously at many places in the brain. There is no supreme perceptive centre, there is no one to have another look at what is happening in the brain and then say what it means, for example, I see a bright object. As to how the principle of organization takes place has not yet been solved. One presumes that certain mechanisms unite the individual stimulations to form an overall picture. The system which produces this overall impression is obviously our memory. Perhaps the solution to the apparent paradox is that there is no supreme perceptive instance in our brain. Although we experience a world, it is represented in many parts of the entire brain. We must assume that our memory is the integrative system which creates our world. Yet, our memory itself is distributed over the entire brain. Here one would have to imagine the normal situation of perception. At the symposium about psychophysics that I have just come back from it was said that one of the most significant problems facing perception is the fact that the sensory organs take in much much more information than can be processed by the brain. The first and most important step here is to eliminate everything which is familiar, that is redundant. The brain looks for what differs, what was not to be expected, what does not result from the context. This is an extremely efficient-way to reduce complexity, whereby the memory must constantly decide: known-unknown, new-old, interesting-uninteresting. The memory joins our perception together to form a materialized whole. All systems are, so to speak, at the service of the memory, which is the relevant state of experience and which depends on previous states of experience. Even before birth, the brain starts to accumulate experience and each experience again creates a new perceptive situation. The result is appraised and is deposited in the memory.
Q: If, as you say, only information that is new and unexpected can be registered by the memory, then one could, to put it naively, not even orientate oneself in our environment which is constituted by expectations? We expect that the next step we take will not have us fall into some empty space. We do look at, even when perhaps not attentively or consciously, objects or spaces which are generally familiar to us, as we could otherwise not move around with any sort of routine and without "knocking into" something. In addition to this, we are often afraid of something new, especially when it is something unexpected. Is the elimination of the familiar really a basic feature of the memory, which has to be memorized in order to be able to compare it with an experience?
A.: Let us consider the situation facing our perceptive system every tenth of a second. If we were to experience everything primarily perceived by our sensory organs, we would never be able to act. But we must act by reacting to important things and by ignoring the unimportant. The most simple possibility of solving this problem is to be found in the fact that the memory itself produces everything it can expect. We subjectively experience a great deal in our visual environment that comes from our memory and that we have actually not perceived. The central perceptive system is only penetrated by what was not to be expected. By means of such a constructive perception one can react in the shortest possible way. A system which always only registers the differences to what was perceived before, before meaning a tenth of a second ago, is exceptionally economic as it is the differences that matter. Everything else is produced centrally by the memory and is "added". For this reason we live in a highly construed world. This is also the survival factor of the constructivity of the brain, as we can produce an exceptionally complex behavior in an exceptionally short time without having to recall an endless amount of data from the environment. If our brain was not so constructive, we would never be able to survive in the complex natural and social environment.
Q.: A strong drive to develop illusion technologies can be observed throughout the history of mankind, starting with rituals, ceremonies, images, as well as the theatre, panorama and so-called virtual-reality, where we put on a "diving suit" and can enter a computer controlled picture. Why do we want to invent doors in order to be able to enter an illusionary and self-created reality? And is it realistically conceivable that we will – at some point in time – be in a position, thanks to brain research, to directly couple the brain to a computer which will be able to release certain stimulations without us having to contemplate pictures or put on a data suit? That is, will we be able to bridge our sensorium and still possess a perceptive world?
A.: We must realize that we have already reached this stage. The world we live in, which we are a part of, is a construed world. The question is only, whether there is a further construed world in this construed world, a world of, so to speak, a second degree. We can, of course, already achieve a lot of effects today with brain stimulations. As to whether the nerve system can stimulate so specifically for us to have exactly those perceptions as we can have with our sensory organs, is a question which still remains to be answered. I don't think so. A further question is also why do we want this at all, because the sensory organs are these exact stimulators.
Q.: We can also supplement them by connecting them to technical equipment, such as a telescope, which also enables us to make sensory experiences that we would otherwise not be able to make.
A.: Your first question was why do we have such a drive to process everything. Even the most simple cultures have this drive to transcend harsh reality. This would appear to be rooted in the fact that our brain is incessantly constructive and is hampered by the sensory data to this overburdening constructivity. We are constantly engaged in construing realities. And we will be made to select one of them which is best compatible with the actual sensory data. If this coupling is done away with, we either dream or hallucinate. This is actually a normal state. It is therefore an elementary pleasure for our brain as a cognitive system, to produce worlds. Perhaps it is rather more unpleasant for us to sort out the very world that best corresponds to the "hard" sensory data.