Abstract

This lecture addresses the creative, cultural and cognitive aspects of symbolic and procedural thinking in contemporary generative art. Generative art is perceived broadly, as a heterogeneous realm of artistic approaches based upon combining the predefined elements with different factors of unpredictability in conceptualizing, producing and presenting the artwork, thus formalizing the uncontrollability of the creative process, underlining and aestheticizing the contextual nature of art. The introduction provides an overview of generativeness as one of the key factors of art making, outlines the characteristics of ludic, pseudo- or proto-scientific methodology that distinguish generative art, and focuses on generative principles in digital art. The main section covers five specific topics in contemporary generative art by presenting the exemplary art projects and commenting on the conceptual, technical and poetic qualities of their methodologies. Transcoding looks at the reflections of early Nam June Paik’s hacking and transmedia imagination in contemporary generative art. Crowdsourced Remix addresses combining the programmed regularity or randomness with the (un)predictability of human input. Distracted Computer Vision addresses the generative use and ‘abuse’ of CV, which opens a new perspective for the critical assessment of traditional visual skills such as selective observation and depiction. Selective Semantics addresses the applied quantification of phenomenological qualities as a theme in itself. Escalation addresses the engaging effects of infinitesimal accumulation and sporadic disruptions of the monotonous build-ups in generative art. The final section summarizes the specifics of symbolic and procedural thinking which is required for the development of generative art projects, and discusses its cognitive implications for creativity in general.

Keywords: abstraction, algorithm, analogy, anticipation, art, code, cognition, creativity, digital art, generativeness, innovation, intuition, invention, knowledge, language, learning, predictability, procedure, programming, reduction, semantics, software.

Generative Art and Digital Art

The conceptions of generative art in contemporary discourse differ by inclusiveness.^{[1][2][3][4][5][6]} In this paper, generative art is perceived broadly, as a heterogeneous realm of artistic approaches based upon combining the predefined elements with different factors of unpredictability in conceptualizing, producing and presenting the artwork, thus formalizing the uncontrollability of the creative process, underlining and aestheticizing the contextual nature of art.

Similarly, the term digital art in this paper denotes a wide spectrum of practices based upon the innovative, experimental, direct or indirect application and exploration of emerging (digital) technologies in correlation with scientific research, which strategically redefine the notions of both traditional and new media, and challenge the distinctions between artistic process, experience and product.

Generativeness

Like all other human endeavors, the arts always emerge from an interrelation between control and accident, and exist in a probabilistic universe. The artists' consciousness about and appreciation of the impossibility to absolutely control the creative process, its outcomes, perception, reception, interpretation and further use is often not their principal motivation, but it becomes central in generative art. Generative art appreciates the artwork as a dynamic catalyzing event or process, susceptible to chance and open for change, which brings it close to the concept of computer science as the 'science of processes'.^[7]

The early examples of planned inclusion of chance in western art, with aleatory music in the fifteenth century, through the musical game of dice (Musikalisches Würfelspiel) and W.A. Mozart's (unauthenticated) instruction-based random compositions in the eighteen century, lead the way to the experiments with indeterminacy in the early twentieth century.

Duchamp's works based upon chance and aesthetic indifference motivated John Cage's exploration of indeterminacy, for example in Music of Changes (1951), in Imaginary Landscape series (1939-1952) and, most notably, in 4:33 (1952) which affirmed the conscious use of chance, interaction and openness to various media as the legitimate artistic principles.^[8]

Contemporary generative art emerges from the mid-twentieth century Modernist exploration of the nature of creative process, of the material, semantic and contextual identity of the artwork, influenced by information theory, system theory, cybernetics and semiotics.^[9][10]

Algorithm and Unpredictability

The use of instructions and language in minimalism and in conceptual art introduced the algorithm and procedure as formal elements but also as participatory factors, e.g. in Sol LeWitt, Lawrence Weiner and George Brecht. It emphasized that the operation of an algorithm, as a structured set of rules and methods, may be well comprehended but its effects can evade prediction.

The cognitive tension between the banality of pre-planned systems antheir surprising outcomes became one of the major poetic elements in minimal music, for example in Steve Reich's opus in the 1960s, with the astonishing effects of phase shifting, iteration, repetition and accumulation of musical figures, in Fluxus, in some process artworks such as Hans Haacke's Condensation Cube (1963), and in some land art projects such as Walter DeMaria's The Lightning Field (1977).

Artwork as a Study: Curiosity and Generativeness

The idea of structuring a complex artwork in the form of research or study is widely adopted in contemporary art, particularly in digital art in which the fluidity of boundaries between production stages (idea, draft, sketch, final work, presentation, reproduction, etc.) is a given.

A forerunner to modern study-based art is Leonardo da Vinci whose career is defined by the intellectual, emotional and aesthetic joy of experimentation, discovery and learning.^[11][12] His exceptional curiosity and need to scrutinize and artfully share his wondering of the world have been highly influential and motivational in the arts, science and technology.

With masterful, visually compact explorations of the paroxysms of facial expression in Characterköpfe, Ausdrucksstudien and Selbstportraits series (cca. 1770-1783), Franz Xaver Messerschmidt is another important antecedent and emancipator of modern understanding of the study-based artwork.^[13][14] The study-based approach which emphasizes the experiential aspects of the artistic process and the experimental nature of the artwork intensifies in Modernism as epitomized in Picasso's oeuvre. The study-based approach was prominent in Neoconstructivism and in early computer art, in which the principle of trial and error was established both conceptually and technologically.^[15][10]

In recent art, Roman Signer's work highlights the idea of ludic, proto- or pseudo-scientific experiment which is performed out of curiosity to actually experience, not just envision, what will happen in certain preconceived circumstances.^{[16][17][18][19]} This idea is important in digital generative art because the computer-based methodology facilitates rapid testing and prototyping, but primarily because it points to the specific qualities of procedural thinking. A notable exponent of ludic, proto- or pseudo-scientific approach in popular culture is Gaston Lagaffe (drawn by André Franquin from 1957 to 1996) who induces warm identification with every child regardless of their age. Always keen for risky experimentation and hacking, often with catastrophic consequences, Gaston is a paradigm of fresh, charmingly nonchalant attitude toward research-based invention, but also toward institutional and social conventions of the 'grown-up' world. He embodies a ludic cognitive drive as a topological layer of creativity which comes before scientific and artistic method.

With a tendency of the art career to become a routine, secured, crafty enterprise, the study-based methodology introduces a conscious risk of failure or waste of energy and resources, thus emphasizing the ethical aspect of art in general. The fact is, however, that even the study-based methodology can regress into a formula or brand for an essentially mediocre art production. This aspect calls for special attention in contemporary art and art education in which 'research' and 'experiment' rank among the most revered keywords.

Generative methodologies in digital art indicate the probable origin of the term generative art in the linguistic concepts such as 'generative grammar' and 'transformational-generative grammar'.^[20] The artists combine the principles of innovative combinatorics with connective and transforming operations such as analogy making and invention. Here are some examples.

Transcoding

A significant legacy of hacking and transmedia imagination in digital art stems from Nam June Paik's early experiments with electromagnetic audio and video signal. Paik's sound installation Fluxusobjekt Random Access (1962-1963) playfully and elegantly deconstructs the dictate of linear succession in musical reproduction and in narrative structures, and resonates with a number of sophisticated contemporary interactive works.^[21]

Paik's Participation TV 1 (1963-1966) in which the audio signal from the microphone explosively deforms a generated circular shape on the TV screen is reflected in Ei Wada's Braun Tube Jazz Band (2008) in which the artist recorded videos by sending various audio signals to the composite video in of a CRT monitor. Then he played these videos on several CRT monitors whose video out signals were amplified as sound and modified by varying the distance of the performer's hand from the screens.^[22]

In Magnet TV (1965) Paik distorted the TV image by placing the magnet on top of the TV set, and in some setups the audience could move the magnet and animate the picture. Carsten Nicolai's installation crt mgn (2013) is a direct reference and tribute to Magnet TV. In crt mgn, the light of four wall-mounted neon tubes is recorded by a video camera and transmitted to a TV screen whose image is distorted by the irregularly swinging pendulum-mounted magnet. The moving magnet induces electricity in an electric coil and the magnetic field fluctuations are transformed into sound.^[23]

Crowdsourced Remix

Things get particularly interesting when the programmed regularity or randomness mix with the (un)predictability of human reactions and decisions.

For his Game Music project (2004-2006), Vladimir Todorović developed a freely downloadable mod for the popular online game Unreal Tournament, which does not affect the gameplay but encodes the actions and conditions of each player and sends them back to the artist's server which converts them into sounds and arranges these into musical compositions.^[24]

The online exploitation of mass participation is addressed in several projects by Aaron Koblin. In Bicycle Built for Two Thousand (2009), Koblin and Daniel Massey crowdsourced an 'analogue' reconstruction of the song Daisy Bell (Bicycle Built for Two) from 1892, whose lyrics were used for the first successful computer voice synthesis in 1961.^[25] The 2088 notes of the song were served to Amazon's Mechanical Turk whose users were tasked with recording their sonic interpretations of the notes, one note per task. The recordings were collected and rearranged back to their original positions in the song. The visitors to the project web page can listen to various versions of the reconstructed song by selecting different human- or computer-generated voice tracks.^[26]

Reface [Portrait Sequencer] (2007-2010) by Golan Levin and Zachary Lieberman wittily exploits the sight as a dominant human sense and the human reliance on facial communication. Its camera captures a brief video of the viewer's portrait, divides the image in three horizontal slices (mouth/nose, eyes and forehead) and dynamically mixes them with the slices taken from previous viewers. The face tracking algorithm provides the continuous capture of the viewer's moving face, the seamless alignment and blending of the video slices, and triggers their rearrangement as the viewer blinks.^[27]

Distracted Computer Vision

Kyle McDonald's brilliant innovations, modifications and applications of real-time computer vision (CV) and face recognition software open the new aspects for critical assessment of the role of selective observation and depiction of details in drawing.

McDonald's and Matt Mets' Blind Self Portrait (2012) uses CV to build a contour portrait of the visitor who has to keep the eyes closed while holding a pen on a moving platform in order to execute the drawing, thus becoming both the subject and a slightly inconsistent 'mechanical part' of the system.^[28] This seemingly passive role is similar to the role of William Anastasi's body in his Subway Drawings from the 1960s.

In McDonald's Caricature (2013)^[29] the CV determines the displacements of moving facial elements in the video image and amplifies them in real time. The most radically moving parts of the face become grotesque instead of the most prominent ones in the traditional caricature.^[30]

As a mathematically formulated detection and interpretation of visual data, computer vision can be delusional. True to Leonardo's view of pareidolia as a device for unorthodox rendering of the known visual forms,^[31] Shinseungback Kimyonghun trick the CV in their Cloud Face project (2012) to select the face-like forms in the video images of a cloudy sky.^[32] Pareidolia in CV, subsequently popularized in Google's DeepDream software (2015), indicates that apophenia^[33] unfolds somewhere between perception and reception.

In Portrait (2013) Shinseungback Kimyonghun use CV in statistically based style of Jason Salavon to generate a series of averaged portraits representing the personalized identities of different movies. The software detects faces in every 24^th frame of the selected movie and blends them into one.^[34]

Human mind is prone to fetishizing for various reasons and in different ways, and CV can be helpful there as well. The White Glove Tracking project (2007) by Evan Roth and Ben Engebreth was a humorous multi-participatory online exercise in tracking Michael Jackson's venerable white glove in 10,060 frames of his landmark live performance of Billy Jean at Motown 25 TV show in 1983.^[35]

When applied to the continuously self-obscuring or self-obliterating visual phenomena, CV reveals their complexity. In Graffiti Analysis series (2010), Evan Roth transforms the four-dimensional events into exciting 3d printed objects by recording the spatio-temporal dynamics of the spray can jet during graffiti writing.^[36] Time is extruded over the Z axis and pen speed is represented by the thickness.

Inmi Lee and Kyle McDonald used similar CV-based method in Mother (2012) to track, isolate and materialize the hand gestures of the people while talking in response to certain linguistic cues such as bbijook, bbejook, bogle, bbogle, bingle and bengle.^[37]

Selective Semantics

In generative art, certain qualitative phenomenological aspects are selectively quantified and turned into something else. This principle can be a theme in itself.

R. Luke DuBois explores the contextual factors of the uncanny allusiveness of selective quantification in a number of projects. In Self-portrait 1993-2014 (2014), for example, he used semantic analysis to visualize his email correspondence over a period of twenty years. It is a force-directed graph resembling a stellar map, with DuBois' primary email addresses in the center surrounded by the addresses of several thousand people. The frequency and tone of communication, and language familiarity determine the attraction forces between the visual representations of correspondents.^[38]

The online profit-oriented automatic recognition of linguistic and behavioral patterns was deftly subverted by Mimi Cabell and Jason Huff in American Psycho (2012). It was created by sending all the pages of Bret Easton Ellis' novel American Psycho (1991) through Gmail, one page per email, and collecting the Google ads that appeared next to each email. The ads were used to correspondingly annotate the original text which was then erased leaving only the original chapter titles and generated footnotes.^[39] Printed and bound in the book format, American Psycho ironically uses the early 21^st century marketing and business strategizing mechanisms based upon data mining and semantic analysis to process (the semantics of) the narrative about the paroxisms of business culture in the early 21^st century.

Escalation

Automatic pattern recognition relies heavily on the frequency of iterations and on the efficient (speedy) accumulation of specific instances. However, even when the accumulation is infinitesimally slow, the effects can be stunning. In Jonathan Schipper's Slow Inevitable Death of American Muscle (2008) two 1978/1979 Pontiac Firebirds are pressed into a head on collision which takes six days. During that time each car moves about 90cm into the other, performing a 50km/h collision at a speed of 0.00000625km/h.^[40]

Similarly, the effects can be engaging when the relatively monotonous build-ups get sporadically disrupted. Sudden changes in sound and image, generated by the random walk algorithm which was modified with cosine function, accelerated and decelerated, determine the overall ambience of Vladimir Todorović's video 1985 (2013). Inspired by George Orwell's 1984 (1949), it is an abstract visualization of the fictional activities of the four ministries (Peace, Love, Plenty and Truth) that govern Oceania one year after the events in the novel.^[41]

Todorović successfully employs this ambiental concept in several other animations, for example in Silica-Esc (2010) which remixes the works of Jean Luc Godard (Weekend, 1967), Velimir Khlebnikov (The Radio of the Future, 1921), Fyodor Dostoyevsky (Notes from the Underground, 1864), and quotes by Caspar David Friedrich. It uses the cloth simulation algorithm for rapid generation and movement of abstract visual elements synchronized with sound and narration.

Beyond Procedure

At least two modes of thinking unfold in the development of generative art projects. One concerns matching the algorithmic and the unpredictable elements into a coherent generative system. It relies on the anticipation or expectation of the performative qualities of the system, based upon experience, knowledge and intuition. Another mode is required for the construction of algorithms as multi-purpose tools.^[9] It requires procedural literacy and programming skills, particularly for generative projects in digital art.^[42] Procedural literacy, understood as "the ability to read and write processes, to engage procedural representation and aesthetics", implies that programming is not just a technical task but an act of dynamic communication and symbolic representation of the world.^[43]

Procedural thinking includes three steps: dematerialization of certain phenomenon into a set of signs that describe it properly, resolving that sign set into pure syntax (removing the semantic layer), and translation of the syntax into a series of operations (elements of the programming language).^[20] This 'trivialization' requires a set of cognitive abilities: intuition or sense for recognizing the phenomenon which can be algorythmized under particular conditions, imagination and flexibility of reasoning, distinguishing between the rational and irrational aspects in the mental concepts of natural phenomena, and attention to scope and inflexibility of the algorithmic (computer) system.

Procedural thinking faces some challenges, especially in creative coding. The conceptual constraints (syntaxes) of programming languages and the hardware architecture can impose certain solutions and unwillingly spin the artistic process.^[43] The fixed performative capabilities of the hardware can reflect in roughness and lack of spontaneity in the simulations of natural phenomena.^[5] Finally, there are the undecidable problems in computability theory, and the limits of mathematical formalization established in Gödel's incompleteness theorems. But protocols, material and formal boundaries are enforced by men or nature to all human activities, not just to procedural thinking. While optimization of productivity and expressiveness within restricted frameworks requires hard mental work, the quality of effort in breaking out of restricted frameworks is the essence of creativity.^[44]

Generative approaches in digital art explore the discovering potential of the plasticity and adaptability in mimicking natural phenomena as the defining factors of universal computing machine.^[45][46] This exploration requires ingenuity, team work, interdisciplinary research, understanding of accumulated knowledge, and learning.^[47] The successful generative artworks are powerful tools for blending the elements of unrelated matrices of thought into the new matrices of meaning through comparison, abstraction and categorization, analogies and metaphors. They tell us stories but, more importantly, they motivate creativity by revealing their background thinking processes in a less abstract and more engaging way than traditional artworks. Similar to computer software, they encapsulate specific intellectual energy which can be engaged implicitly or explicitly and incite original, often surprising, configurations and ideas.

The examples in this paper are distinguished by the artists' abilities to expand the potentials of generative methodology by creating new contexts which transcend the conceptual, productive, expressive and aesthetic limits of code-based art making. They elegantly demonstrate the amazing capacity of human mind to simultaneously invent the technology, absorb it, adapt to it, repurpose and transform it as needed.