Ars Electronica 1992
Festival-Program 1992
Back to:
Festival 1979-2007


Our Rainbow World

'Peter Weibel Peter Weibel / 'Otto E. Rössler Otto E. Rössler

A rainbow's ends stand in a pot of gold, it is said. Its location indeed poses a problem since it is different for every observer. The rainbow actually is a distorted virtual image of the sun. Nevertheless it looks like a real object. Could it be that similar distortions apply to other "real" objects?

To what extent does objective reality depend on the observer? Since the invention of perspective in the Renaissance, and the invention of group theory (Helmholtz-Lie groups) in the nineteenth century, we know that the appearance of the world depends on the location of the observer in a lawful manner. Computer programs of the "virtual-reality" type accordingly generate a "lawfully non-invariant" (that is, covariant) representation out of an absolute (invariant) one that is present in the computer memory. Even though the lawful distortion of perspectivic vision is tantalizing, it leaves our secure sense of an existing "objective" reality undisturbed.
The rainbow challenges this security. Virtual-reality programs containing rainbows have so far not been developed. The transformation rules are different than those for other objects. The reason has to do with the fact that a rainbow is a rather special kind of object: it is a distorted virtual image of the sun. Thus, if the observer is travelling, so is the rainbow. If the observer artificially increases the distance between the eyes by the use of mirrors (which can be mimicked in a virtual-reality simulation by changing the size of the internal representation of the observer), the rainbow consistently keeps an infinite distance, despite the fact that it is overlaid over rather closer-by objects. While no one doubts that a pot of gold is waiting at the foot of the rainbow, it is somewhat difficult to simulataneously stay and watch and sneak toward the right place to catch the pot.

Thus, the properties of certain objects (in the present case: their location) depend on properties of the observer (like the latter's location and pupil shape) in a way which goes beyond the familiar distortions of 3-D Helmholtzian perspective or 4-D Minkowskian projection. Could it be that the principle illustrated by the rainbow is of a broader significance?
An observer who is part of the world cannot see that world from an objective vantage point. The homogeneous matrix algorithm of the flight simulators of virtual-reality (cf. Newman, Sproull, 1979) shows how non-trivial a task it is to generate the right interface. The observer, far from being intimidated by the rich and changing structure of the sequentially applying perspectives, actually needs this kind of packaging in order to extract from it the correct, invariant representation. "The intimacy of a head near one's own is like the lights and doorway of a house" (Rodney, 1991). In principle there are many more parameters to try out than those of observer location and size. Motion of the observer comes to mind immediately. Both "visual flow" phenomena and relativistic distortions are hereby generated and can indeed be reproduced simulationally (Sutherland, 1966, 1970). Next, take recurrent motions of the observer, like a shaking of the head. The effect on the interface can be dramatic, especially if the shaking is fast. Indeed, irreparable damage can be done to the goal of finding an invariant representation under such a predicament.

Historically, the interface problem was first seen by Boscovich (1755) who asked what happens when both the observer and the surrounding world were shrinking concomitantly, along with of all involved forces. Obviously, "the same impressions would be generated within the mind." The interface would remain unaffected. Similarly, if the shaking of the observer's head is accompanied by a matching shaking of the rest of the world, nothing is happening for the observer. Therefore, time-dependent features of the interface deserve a closer look.
Brownian motion or "Archimedean motion" is interesting because of the energy and momentum conservation involved. Any observer who is built out of particles that are in random thermal motion stands in an interesting dynamical relationship with the rest of the world. Archimedes first saw that the joint "center of gravity" can never be moved. How, therefore, does the rest of the world appear to such an observer? This question makes sense to ask only to date since the necessary simulation of many particles simultaneously is a fairly recent option (Alder and Wainwright, 1957).

Every external object will be found to be performing a Brownian motion relative to the observer. The strength of this motion will be dependent on the object's mass: The smaller the mass, the larger the apparent thermal agitation. This is because the center of gravity of the observer and the external object are linked by a relative Brownian motion. A very small mass object can therefore never be observed accurately by a thermally trembling observer. The thermal noise of the observer will always "infest" the object in such a way that it is the latter which appears to be thermally agitated by the temperature of the observer, even if the object's actual motion temperature was zero.

The effect is the same as if the observer was a Brownian particle himself or herself. How does the world look like to a particle in Brownian motion? The virtual-reality paradigm can in principle be used to find an answer.
Finding the right tranquility of mind to try in the right way is not easy. Numerically, the task is also very demanding. What is needed is to design a whole reversible micro world in a computer. The "eye" inside (that is, the internal macroscopic observer) is to be built out of the same micro constituents as the rest. The special thermal (momentum-conserving) relationship between that "eye" and a particular micro object, valid in the computer universe, can then be looked at by a human macro observer outside that world (if he wears the right kind of goggles). It will be rewarding to implement this task say in the year 2010, but even today it is already possible to glimpse some of the unusual rainbow phenomena that will emanate from this contraption.
An irreducible uncertainty is a first implication. The chaos in the observer translates into chaos outside the observer. Apart from the unit thermal noise energy inside the observer (E), which is equal to one half the Boltzmann constant times the temperature of the observer, we have a second intrinsic constant (M). This characteristic time interval is related to the mean collision interval inside the observer: After this time interval has passed, the micro dynamics inside the observer changes course relative to the external object. A precise calculation of T for classical billiard systems is an open problem (Rössler, 1991 a). The mean shaking period T needs further clarification from a conceptual point of view as well. If the observer was alone in the universe with the object, the center of gravity of the observer and that of the object would not perform a Brownian motion relative to each other. However, as soon as any third object (for example, a mediating particle) is coupled to the observer, it is only toward this combined set that the external object remains in a state of constant motion. The multi-particle observer now indeed in general reverses course, every unit time internal T, relative to the external object.

The resulting "relative diffusion" between external object and observer is governed by the product of E and T, divided by the object's mass (M). This result holds true when the external object is "directly" (that is, without a measuring chain) coupled to the observer (Rössler, 1987). Unexpectedly, the more general case of "indirect" coupling (via a measuring chain) is still governed by the same law since the measuring chain is unable to undo the objectively existing mutual relationship between observer and object.

The resulting "uncertainty" mimics quantum mechanics. This is because the presence of a diffusion law of the same qualitative type as described above (an action – like E times T divided by the object's mass) is sufficient to generate the Schrödinger equation (Fényes, 1952; Nelson, 1966).
We still need to know what happens when the observer forces a micro object into a certain definite observational state. For example, the measurement situation may be chosen such that the micro object must reveal its position in a yes-or-no decision. The problem on hand is analogous to the problem of the formation of an "eigen state" in quantum mechanics. Such a restricting type of measurement can certainly also be performed in our simulated world.

Here a new phenomenon arises. While the previous finding (uncertainty) did not yet qualify as a rainbow phenomenon in the strict sense since mere blurring does not bring in a new phenomenological quality, in the present case a new quality emerges. It is the quality of a well-defined localization in position space (or momentum space, respectively) appearing for the observer which is at variance with the correct location. For if the observed location of the object were identical with the correct location, the relative Brownian motion of the observer would have been eliminated in effect even though this cannot happen. Therefore, the apparent location of the object, valid in the interface, is different from the objectively applying location. This prediction can be verified in the proposed simulation of the interface. Since everything that happens in the simulation is known explicitly, it is possible to compare the content of the interface with what really happens to the particle in question. This comparison is, of course, a privilege confined to the external operator since the internal observer is stuck with the interface.

The yes-or-no decision which appears on the interface depends on the internal dynamics of the observer as much as on the object's. According to Nelson's (1966) stochastic mechanics, that is, diffusion theory, the probability of a certain decision occurring depends on the square of the amplitude of the diffusion-generated Schrödinger equation. This diffusion-theoretical result can be expected to be confirmed once the first simulation of the interface becomes available. However, there is a "complication" to be expected in that case which is absent in the standard formalism of stochastic mechanics. In the latter, the occurring decisions ("eigen states") are assumed to be permanent. Here, the distortion of the objective world is such that the recorded state, as it appears in the interface, depends on the momentary state of motion of all particles inside the observer. In other words, the interface is a momentary state of affairs. All measurements, no matter how long the measuring chain in terms of space and time, are determined by the momentarily valid relationship between the internal dynamics of the observer and the dynamics of the rest of the world.

An external super-observer who watches the momentary interface as a function of time will therefore record a "superposition" (that is, a temporal integration) over all the momentarily valid "quantum decisions". The momentarily valid "eigen worlds", while mutually different, all fall within the probability distribution prescribed by the wave function of stochastic mechanics.

In quantum mechanics a similar problem is known under the name of "the measurement problem". For example, in the language of Everett's (1957) "relative state" formulation, the different eigen worlds that apply at every moment are said to be "shielded" from each other. There exists one version of Everett's formalism (due to Bell, 1981) in which the different eigen worlds are assumed to exist, not simultaneously as in the usual Everett picture but sequentially – each confined to a very small time window. Bell only wanted to show the mathematical equivalence of this view with the standard, multiple-worlds interpretation. Both interpretations of quantum mechanics are usually considered rather outlandish. Here, the second interpretation unexpectedly arises again in a quite different context.

Bell's insight that the observer would "not notice" being in a different quantum world from one moment to the next (since worlds by definition are complete, that is, contain no trace of another world) is applicable here as well. It follows that the "integration" which an outside observer of the simulated interface experiences is an artifact.

If the outside human observer were a part of the same interface, being unable to escape from it through the use of an outside memory, the phenomenon of integration would disappear and a single consistent "eigen world" would apply at every moment, complete with its own recorded past and anticipated future. Thus, the job of a demiurge – to notice the implications that his own actions (laws and initial conditions) generate for the inhabitants – is surprisingly hard.
The distortion of an objective world as it is mirrored in an interface thus can go unexpectedly far. The notion "rainbow world" applies to each distorted representation no matter how short-lived. In the one world, for example, Schrödinger's cat is alive and well while in the other, the same "hellish contraption" (Schrödinger, 1935) has chosen the other course. Moreover, that same branching may have taken place some while ago, so that the one outcome would have produced a cat that is playful and frisky right now while the other entails a cat that has been subjected to organic decomposition for quite a while. It appears very hard to reconcile both rainbow worlds with one and the same exo reality.

Equally hard to accept is the claim that these two different internal refractions of the same objective reality alternate at a rapid pace in an unnoticeable way. This "rainbow movie" (one time slice after the other) accordingly contains many consistent "sub-movies" of which a different one is in charge at every moment.

The counter-intuitive notion of a rainbow movie needs further scrutiny. One of its features, however, unexpectedly is very close to everyday experience. It is the fact that each moment has its own world (eigen world). In quantum mechanics, the same 1:1 relationship was noted by Deutsch (1986). Here, the same result arises in a completely transparent context (provided all difficulties have been mastered). The inhabitants of a reversible universe are strangely glued to a single moment in time. They call it their world "as it is real now". While the mutual incompatibility of the different "now worlds" lacks a representation in the interface as mentioned, the interface still gives away the fact that a single instant in time is privileged over all other because it "defines a world".

The latter prediction – existence of a now-world for internal inhabitants -is, when transplanted back to our own world, at variance with traditional science which lacks the notion of a privileged now.

The paradigm of virtual reality has made the topic of the "interface" scientifically acceptable (cf. Ars electronica, 1986; Weibel, 1990). The momentary position of the camera distorts the world in a way which makes it fully palpable as an invariant new reality. Generating such an interface is not easy and requires a lot of computer processing power. Experimenting with this interface is presently an important technological and conceptual challenge. How, for example, does a rainbow look inside when it is "reduced" by the vertical pupil of a cat rather than by a round one? How about a cat's pupil that is many meters long, either vertically or horizontally?
A second novel question refers to temporally changing realities if the changes occur in both the position of the "eye" and that of the external object in a correlated fashion. Such changes will obviously not show up in the interface (since the "Boscovich difference" is zero; Rössler, 1991 b). Third, there is a very special interface, generated between an microscopically described reversible observer and the rest of the same microscopically simulated world. Phenomena only known from the counter-intuitive realm of quantum mechanics suddenly arise as implications of a conceptually completely transparent situation. At the same time, "nowness-bound rainbow worlds" become a topic for scientific discussion.

Thus, "playing with cameras" can be a rewarding pastime. Diverse phenomena known from everyday experience can be retrieved. At the same time a new type of suspicion regarding our own world arises: Maybe, our own world is a rainbow world, too?

Once such a suspicion has taken hold, the logical next step is to call for new diagnostic tools that can be used in our own world to demonstrate the existence of the new predicament and to explore and perhaps manipulate it. Nevertheless, the decisive step is getting suspicious in the first place. The present suspicion, which goes back to Kant and Boscovich, and before them to Anaximander, has now found a new medium for its study.

To conclude, the concept of the rainbow has been re-examined from the vantage point of virtualreality simulations. A rather unusual type of virtual reality is needed for such a simulation. Eventually, reversible simulated worlds will be useful to further the understanding of the human / world interface (a proposal which at first sight is confined to the study of an ice skater who cannot get rid of whole-body angular momentum, or of an Archimedean system of interacting balls and springs like a model drug molecule). The first detailed report about the properties of such a "conservative virtual-reality" will come in in about ten years time. Presently, only "informed guesses" are possible. In this way, a new "hopeful suspicion" could be arrived at: The VIR paradigm may reveal more about our own world than the ordinary course of science has prepared us to believe. For example, the walls of the prison of the now become palpable. Further distortions of the invariant (exo) reality may exist which can likewise be unmasked by the new Hermetian paradigm of computer-generated worlds.


ALDER, B.J., WAINWRIGHT, T.E., "Phase transitions for a hardsphere system", Journal of Chemical Physics, 27,1957, p. 1208.
Ars Electronica 1988, Philosophies of the New Technology / Philosophien der neuen Technologie, Berlin 1986.
BELL, J.S., "Quantum mechanics for cosmologists", in C.J. Isham, R. Penrose, D. Sciama, eds., Oxford, Clarendon Press 1981, p. 611-637.
BOSCOVICH, R.J., "De spatio et tempore, ut a nobis cognoscuntur (On space and time, as they are recognized by us)", in J.M. Child, ed., Boscovich, R.J., Theory of Natural Philosophy, Latin-English Edition, Open Court, Chicago 1992, p. 404-409. Reprint of the English translation: MIT Press, Cambridge, Mass. 1966, p. 203-205. English re-translation in Rbssler (1991b). Compare also: Fischer (1991).
DEUTSCH, D., "Three connections between Everett's interpretation and experiment", in R. Penrose, C.J. Isham, eds., Quantum Concepts in Space and Time, Clarendon Press, Oxford 1986, p. 215-225.
EVERETT, H., III, "Relative state formulation of quantum mechanics", Review of Modern Physics, 29, 1957, p.454-462.
ONYES, I., "A probability-theoretical foundation and interpretation of quantum mechanics" (in German), Zeitschrift für Physik, 132, 1952, p. 81-106.
FISCHER, R., "A neurobiological re-interpretation and verification of Boscovich covariance, postulated in 1758", Cybernetica, 34, 1991, 95-101.
NELSON, E., "Derivation of the Schrödinger equation from Newtonian mechanics", Physics Review, 150,1966, p. 1079-1085.
NEWMAN, W.M., SPROULL, R.F., Principles of Interactive Computer Graphics, 2nd ed., McGraw-Hill, New York 1979.
RODNEY, T., The Runaway Soul, Farrar, Straus and Giroux, New York 1991.
RÖSSLER, O.E., "Endophysics", in J.L. Casti, A. Karlqvist, eds., Real Brains, Artificial Minds, NorthHolland, New York 1987, p. 25-46.
RÖSSLER, O.E., "Four open problems in four dimensions", in G. Baier, M. Klein, eds., A Chaotic Hierarchy, World Scientific, Singapore 1991 a, p. 365-369.
RÖSSLER, O.E., "Boscovich covariance", in J. L. Casti, A. Karlqvist, eds., Beyond Belief: Randomness, Prediction and Explanation in Science, CRC Press, Boca Raton 1991 b, p. 6987.
WEIBEL, P., "The two levels of reality, exo and endo", in Y. Shikata, ed., Art Lab 1st Symposium, The Current Condition and the Future of Digital Art, Canon, Tokyo 1991, p. 16.
SCHRÖDINGER, E., "The present situation in quantum mechanics" (in German), Naturwissenschaften, 23, 1935, p. 807-812; 823828; 844-849. English translation: Proceedings of the American Philosophical Society, 124, 1980, p. 323-338.
SUTHERLAND, I.E., "Computer inputs and outputs", Scientific American, 1966, September issue, "Computer displays", Scientific American, 1970, June issue.
WEIBEL, P., "Virtual worlds: The emperors new bodies", in G. Hattinger, M. Russel, C. Schöpf, P. Weibel, eds., Ars Electronica 1990, vol 2, VeritasVerlag Linz, Linz 1990, p. 9-38.
WEIBEL, P., B. O'KANE, "Das tangible Bild Endoapproach to electronics", in M. Waffender, ed., Cyberspace, Rowohlt, Reinbek 1991, p. 154.