|
Image Processing and Video Synthesis
Electroniv Videographic Techniques
'Stephen Beck
Stephen Beck
I. ELECTRONIC IMAGING TECHNIQUES. These techniques as applied to television utilize the inherent plasticity of the medium to expand it beyond a strictly photographic/realistic, representational aspect which characterizes the history of television in general. A wide variety of electronic instruments have been constructed by engineers, artists, and engineer-artist collaborations in the past several years which operate specifically with TV sets as primary display or "canvas." Each imaging system which has been developed reflects the artistic and technical capabilities of its originators, and tends to be utilized according to distinctly different aesthetic theories. In some cases the resultant image is largely due to the inherent circuit designs of a given instrument. In other cases, the instrument is utilized to produce an image with a specific visual or psychological effect, the electronic aspect being more of a means than an end to the realization.
Regardless of the specific aesthetic usage of particular instruments, some insight may be obtained by examining the structural differences and similarities between typical video synthesizers and image processors, as well as some of the basic circuitry which is used in these devices. In every case, the video synthesizer may be viewed as a "tip of the iceberg" of electronic technology & visual arts. Vast armies of individuals make the transistors, resistors, capacitors, and integrated circuits which comprise a synthesizing instrument, when properly applied under the design of visually inclined electronic artists.II. CATEGORICAL DISTINCTIONS OF ELECTRONIC VIDEO INSTRUMENTS. Just as in the science of biology, where many classifications of lifeforms exist, there are several genre of video synthesizers. In the sense that a synthesizer in general is something which combines parts to form a whole entity, just about all video instruments could be classified as such. However, in terms of structural details, some clarification can be made. I have listed: several categories of video image instruments according to the unique qualities of their principle of operation, some criteria for making the distinction, and artists and engineers in the video art field who are using these methods.
1. Camera image processor types.
These types include such techniques as colorizers which add chrominance signal to black and white (monochrome) signal from TV camera; keyers and quantizers which separate value levels in a scene and allow other processes to take place in that scene, add synthetic color, place another image in certain places of the original, obtain matte effects; modifiers which do not alter the geometry of the image but rather affect its grey scale, such as polarity inversion, or which generate an edge around elements of the image, or which mix by superimposition several image sources. Systems that are essentially of the image processor type described include those built by Paik/Abe, Siegel (CVS), Templeton, Sandin, Hearn, Vasulkas, and others.
2. Direct Video synthesizer types.
These types are in principle conceived to operate without the use of any camera image, though some of them can also perform the processing operations described above. Basically, a complete TV signal is formed from electronic generators which comprise the synthesizer circuits, which include circuitry such as color generators which produce chrominance signals according to either I-Q methods, Hue-Saturation methods, or Red-Green-Blue methods; form generators which establish the necessary pulse vibrations to produce shapes, planes, lines, or points and to move them in various ways by use of motion modulators with either simple electronic waves such as ramps, sines, or triangles, or more complex curves, or even with audio frequency sound signals; texture amplifiers which allow for color manipulation to achieve shading, chiaroscuro, "airbrush," or granulated effects, roughly could be thought of as electronic brush effects. Instruments using the Direct Video process include those by Beck (Direct Video Synthesizer), Siegel (EVS), Dupouy (Movicolor), EMS (Spectron), and others.
3. Scan modulation/Re-scan types.
These rely on the principle of a TV camera viewing an oscilloscope or television screen which displays the image from another TV camera. The image on the screen can then be manipulated geometrically (stretched, squeezed, rotated, reflected, etc.) by means of deflection modulation, either magnetically or electronically. The second TV camera then transforms this image into one bearing a proper TV scan relationship, and may then be colorized or processed by techniques outlined in section 1. These systems can also be used without an input camera where the image then consists only of the manipulation of the raster, producing Lissajous type images. Systems using this method include those by Harrison (Computer Image), Paik/Abe, Rutt/Etra, and others.
4. Non-VTR recordable types.
These types are included for completeness and encompass those video displays which do not actually produce a standard TV signal waveform and can hence only be utilized on one set which is specially prepared, and cannot be directly recorded on magnetic video tape. Most are based primarily upon magnetic distortion of the normal TV scan pattern, or else they utilize a color picture tube as if it were an oscilloscope screen. Such individuals as Paik, Tadlock (Archetron), and Hearn (Vidium) have utilized these techniques in their video sculpture.
I have not included in this categorization the studio switcher and special effects generator to be found in most teleproduction studios, which include processing and wipe generation, or the emerging video game box which is in principle a direct video signal generator of very specific configuration. Nor have I alluded to video feedback techniques, which all systems are capable of sustaining in one of its various forms.
In every case the individual approach to video instruments encompasses a wide variety of circuit designs and processes. Some require cameras, others do not; some utilize a form of voltage control which permits color changes, image size, or movement rate, for example, to be changed by some other circuit, in addition to being changed by an operator. This factor introduces an interesting dilemma into the realm of electronic images: how much is the image a product of the instrument rather than of the instrumentalist?
A video synthesizer can be set to conditions which generate image after image for hours upon hours-perhaps interesting, perhaps not – depending upon the viewer. But in this case the images have their composition in the circuit design and programming of the instrument. Or the image may be altered and shaped temporally by someone playing the video synthesizer, in which case the images have their composition in the mental image of the player, interacting with the circuit design.
One can conceive of a synthesizer as a generative device which forms the resultant picture by a process of assemblage of electronic pulsation, or one can conceive of it as a filtration device in which, due to the proper selection of numerous electronic conditions, a given image out of the infinity of possible images results as a picture. Giordano Bruno in his thesis "De Immenso, Innumerabilius et Infigurabilibus" postulates an infinite number of universes which are perceived by a selective process to form a reality distinctly unique to the viewer. Thus it is that a video synthesizer and Marconi Mark V color studio reveal very different images-each is filtering according to very different criteria-neither one more or less valid.
When visual literacy has advanced sufficiently, many will no longer consider the synthesized image as a by-product of television technology, but as a visual reality of its own, distinct from the terms of a representational, photographic image, an image which is more glyphic than literal.III. TWO EXAMPLES OF VIDEO SYNTHESIZER CIRCUIT STRUCTURES. In order to ilIustrate in more detail some typical electronic techniques utilized in video synthesizer and image processor circuits I shall mention the comparator circuit and colorizing techniques.
The comparator is a very general circuit used in keyers, quantizers, wipe generators, and form generators. It is symbolized electronically as a triangle enclosing a question mark:
There are two inputs and one output. The inputs can be continuous voltages from, say, a scale of 0-10. The output however is allowed only two conditions: ON or OFF. The appropriate condition is determined by comparing the values of the two inputs. If the + input value is greater than the – input value, the output is ON; if the + input is the same as or less than the – input, the output is OFF. A typical circuit used for this function is the u710 integrated circuit, about the size of a dime.
When the continuous voltage to one input comes from a monochrome TV camera, the value 0 represents any black areas in the image, while the value 10 represents the brightest white areas in the picture, with value 5 representing an area of medium gray. Imagine the image to be a white cross inside a gray square surrounded by a black background. The image could be depicted schematically as
00000000
00555500
00599500
00999900
00599500
00555500
00000000
If the other input to the comparator comes from a fixed value source, called the threshold, then the resultant circuit is a simple keyer. The output will be OFF whenever the picture element is less than the threshold and will be ON whenever the picture element is more than the threshold.
For example, the white cross could be colorized by setting the threshold to say value 7 and connecting the output of the comparator to a colorizer activated circuit. Only where the picture value exceeds value 7 will the color be turned on, in the region occupied by the cross. If another comparator were introduced with its threshold set to value 4, then the output would be ON in the region occupied by the gray box and the white cross, and it could be used to control a second colorizer producing a colored square, which might be combined with the colorized cross. If the two inputs to comparator 2 were exchanged, then the color would be inserted into the area surrounding the gray square.
Clearly this example can be extended to many channels, 8 or even 16 not being uncommon, and forms the basis for quantizing colorizers and multiple level keyers used by some video artists. Bear in mind that the scanning process traverses each line of picture elements in some 52-millionths of a second, with each element being occupied for only 250 nano seconds (billionths of a second) so that comparison must be performed very fast. The u710 can make a comparison in less than 20 nano seconds. But at this high speed, and when the picture and threshold levels are almost equal (within a few thousandths of a volt) the output often is indecisive, oscillating back and forth for a time, producing the speculated or "torn" edge characteristic of keying.
Colorizing.
In television color, three types of phosphors are applied to the inside surface of the picture tube; each emits a different color light when excited by electrons scanning over them. The three colors are red, green, and blue, and are applied in either triadic clusters of tiny dots, or in very thin strips, so that at normal viewing distances the individual phosphors are not discernable as such, but tend to fuse their colors according to the subjective process of color vision. Each of the primary colors can be varied in intensity from zero to 100% by modulating the intensity of the electron streams exciting them. In this manner, polychromatic reproduction is achieved by controlling the admixture of three primary colors. Since the color process is additive and involves the mixture of emitted light, all three colors when excited in equal amounts produce the sensation of white or gray values. When just the red and the green colors are stimulated, a yellow color is sensed, or when red and blue are excited, purples result.
The three properties of color include hue-the wavelength of the color (i.e. yellow as opposed to green or blue); saturation – how intense or vivid is the hue; and brightness or value – how much is the color diluted or made pastel by the addition of white or gray. Any video colorizer must determine each of the three properties. In black and white television (more properly known as monochrome) the picture is composed entirely of various intensities of light of a bluish-white nature. This signal is known in television terminology as the luminance signal. It conveys information of values. With color television an additional information bearing signal is used to convey the hue and saturation information, called the chrominance signal or chroma.
This chrominance signal is present in the form of a color subcarrier which vibrates at 3,579,549 cycles per second. Its intensity or amplitude is varied according to the saturation of the color, and its phase is varied according to the hue of the color. This technique of phase modulation requires the presence of a pilot or reference signal to supply the phase angle reference, known as the color burst.
In essence, the color spectrum maybe visualized as occupying a circular distribution. The center of the circle represents no saturation, while any distance outward from the center represent progressively more saturated colors with the direction representing the hue of the color. In fact there are actually two elements of the color subcarrier which can be controlled to produce synthetic color; the I and Q components, standing for inphase and quadrature.
The simplest colorizers operate on the huesaturation principle, with one control affecting the phase of the color subcarrier and thus determining red, yellow, green, cyan, blue, or magenta hues, while the other control affects the amplitude of the subcarrier to determine the vividness or saturation of the desired hue. An additional control may be added electronically to an existing monochrome signal derived from a camera.
Another type of colorizer operates by modulating the intensity of the I and Q subcarrier components. The combined effect of two independent modulations generates both hue and saturation information, with the two variables being affected simultaneously. Thus to change the vividness of a given hue, both controls must be changed together.
A third type of colorizer circuit is the Red-Green-Blue encoding method. Three controls determine the saturation levels of red, green, and blue primaries, which then mix in the encoder to produce luminance and chrominance signals of the standard video signal. Besides operating in a graphic mode, this type of colorizer is readily adaptable to other TV systems in use by substituting the encoding circuitry. The I/Q and Hue/Saturation methods normally require different techniques for each type of television system used.
Many colorizers are limited to full screen color or quantized color type of operation. This allows for basically hard edge color. In the Beck Direct Video synthesizer I have been particularly interested in surmounting this limitation and achieving a full range of color contouring.IV. VIDEO SYNTHESIS AND COMPUTER GRAPHICS. In the strict sense of the word, a digital computer is but a large collection of electronic switches arranged to operate on binary bits of information. As such, most video synthesizers do not qualify as computers; although the analog computer, with op-amps, differentiators, integrators, and amplifiers more closely resembles the structure of video synthesizers. Computer graphics generally has been done with oscilloscope displays under computer control, though some newer systems do generate images on a color television display directly. We can expect to see the use of digital computers in the control of video synthesizers via means such as digital to analog converters. When one compares the bandwidth of video images (4,200,000 cycles per second) with computer processing speeds (500,000 bits per second typically) or with audio signals (20,000 cycles per second) the gap between computer output speeds and the necessary information rates to generate a moving video image becomes apparent.
In terms of circuit devices, most video synthesizers and image processors utilize discrete transistors and some types of integrated microcircuits. We can expect to see the emergence in two or three years of video-integrated circuits designed specifically for the imaging functions of television display.
Video synthesizers consume electrical power of from 50 watts to several hundred watts-far less than even a single spotlight utilized by the dozen in standard camera studios. They also require far fewer personnel to operate them when compared to standard teleproduction. Both of these factors make video synthesized television images appealing from an economical perspective.V. The appearance of electronic imaging instruments such as the video synthesizer and image processors ushers in a new language of the screen. Non-representational and departing from the conventional television image, these methods will stimulate the awareness of new images in the culture. Any growth of the video-synthesized image will be contingent on the ability of video artists to become proficient in techniques of composing and presenting synthesized imagery. The instruments themselves will not perform without the artistic consciousness of a skilled operator.
|
