Four computers / game engine / hardware / sound / custom code / text/

Mary Flanagan’s interest in virtual environments and interactive writing led to her focus her new work [xyz] on the spatial metaphors inherent within virtual systems and on the grammatical and lexical notions in language itself.

In [xyz], the rules of game playing and the rules for language reside in the same location. Player-readers participate in the dynamic combination of new texts using the fundamental metaphoric system that governs the development of computerized spaces—namely, the 16th Century three dimensional Cartesian coordinate system, with axis lines x, y, and z.

The gallery contains a computer-controlled application for each axis. Each of these directions contains a different section of a larger text. Visitors to the gallery may interact with the words on the screen using the controller located under the screen and collect sets of words as they wish. These words are then sent to the projected image where the player-readers’ choices combine.

A work of electronic literature by Mary Flanagan
Game programming and network design by Jack Bowman
Additional audio by Michelle Earhart

Background to the work [xyz]

There is an ancient desire to expand the expressiveness of poetry by combining literary and visual communication. Shaped poems in Greek, Latin, Arabic, Hebrew, Chinese, and Sanskrit all indicate a common human urge to compel language into the form of its meaning. Visual poetry, or concrete poetry, is the term applied in the 1950s, long after early Twentieth Century visual poet Apollinaire manipulated his text to fashion a literary syntheses of space. If Apollinaire and other writers endeavored to mould pictorial strategies onto the demands of poetry, then Flanagan’s contemporary project intends an analogous mapping—that of applying the metaphors and structures of computer games onto – and into— a poetic system.

The artist’s curiosity about the process of writing and rewriting the text led her to examine the ritualistic behaviors constituting conventional game interaction. In [xyz] the player-reader reconstructs the poem by following a visual logic similar to that used by the player of a platform game. Instead of collecting coins or avoiding enemies as in a stereotypical game of this sort, the player-reader collects and avoids text. In the end, the work changes incredibly through reader participation.

[xyz] functions on multiple levels as a concrete poem, a game, and as way to explore the epistemological development of spatial deixis in a game-like system. Often in games, as in interactive literature, player-readers experience the phenomenon wherein understanding the meaning of certain words and phrases is dependent on contextual information. In [xyz] the reader’s experience of the line becomes the point of reference, the center of the spatial understanding of each text and the site for spatial deixis.

Relationships between the line, interaction, behavior, and metaphor are at play as the player-reader traverses the poem space. How does the player choose to construct his or her mental model in this work?

We use text so frequently in digital communication. Most technology users correspond in written word more frequently than in person conversation or telephone chats. However, we seldom stop to consider “voice” within our mundane, written correspondences.

[ineffable] is a computer application which reads aggregated emails between two correspondents and maps the use of language through the words users utilize in everyday correspondence. The project explores the question, Do we have particular “voice” in our daily writing to friends and colleagues, and does that voice change depending on who we are writing to and why?

Collaborators Mary Flanagan and Andrew Gerngross work to bring into the foreground the primary form of exchange practiced by many technology users — email — through closely examining the use of language in email systems. Email is used for work and play, intimate exchange and legal agreements. How are different kinds of language, and thus sounds, used in correspondences with different people? How do we “sound” to those reading our emails, and how does the email of others sound to us?

Words, phrases, and sentences represent a time, a person, a map of interpersonal experiences (the external world) as well the way a user relates to the context of digital communication — and relationship of that person to their own computer. This project maps the geography of these relationships with sound and image.

The [ineffable] program explores the relationship between real people through their written messages to one another, but further, the work generates a set of procedures which expose how our everyday experiences with email encode the kinds of sounds we make, our “voices,” into our digital systems.

HOW [ineffable] WORKS.
[ineffable] collects chronological information, time between emails, length of correspondence, and most importantly, the kinds of phonetic sounds used by a correspondent in his or her writing and generates a sonification and visualization of this content. Written in Java, [ineffable] analyzes a user’s emails to or from a particular person and maps the language used by examining the phonemic makeup of the words utilized in the correspondence. The work “reads” a pair of user’s email archives and, side by side, analyzes the words therein, grouping them based on the recipient, date, time elapse between correspondences, overall amount of correspondence. Most importantly, we have the program find a “sound signature” to the words used in words, paragraphs, sentences and finally, the overall email set.

[ineffable] functions as an experimental system which considers the “sound in the head” we create while reading and writing as a synaesthetic experience. The visitor to the work encounters a set of dynamic sonifications and visualizations from each set of correspondences in a split stereo sound environment.

Visualizations. The two sides of the visual aspect of the project represent the application’s reading of the email aggregate. Visual modes switch between a moving line map and a “voice organism” color collage generated by the system. The moving line map illustrates sound structures encountered by the system. It shows the frequency of the phonemes created by the email in its thickness and color. The voice organism visualizes time and the change in voice through time. It increases and decreases in complexity and transparency depending on the dates in between correspondences. The smoothness is the “consistent” state of the “voice” or the general phonemic sounds used by the system. Changes in the sound “voice” change the organism. The system generates “ideal words” that are the aggregate of a user’s most frequently used sounds. The most commonly used sounds –for example, if a writer uses many “mmm”s and “er”, generates a representative word from the sounds (in this example, the word could be “murmur”). These are displayed under the visualization and change over time.

Sonifications. In the audio aspect of the work, several audio tracks are created in response to the email data. First, phoneme sounds are assigned to an “instrument” created in the programming API Jsyn. Phonemes sampled from human voices are also played back in the rhythm of the email’s syllabic structure.

PRIOR EXHIBITION OF THIS WORK.
[ineffable] is a new work and was selected by an international jury to premier at the prestigious SIGGRAPH computer graphics conference in the Art Gallery in August 2004.

INSTALLATION DIAGRAM.
This work can utilize a monitor or video projector in its installed form. It runs off of one laptop, with a mixer and 4 speakers (and optional headphones).

ABOUT THE ARTISTS.
The project is a collaboration between New York artist-scientists Mary Flanagan and Andrew Gerngross. Flanagan’s computer and net art has been shown at the Whitney Biennial (2002), the Guggenheim Museum (2004), SIGGRAPH, and Ars Electronica. Her books include Reload: Rethinking Women + Cyberculture (MIT Press 2002) and reskin (forthcoming). She is the creator of “The Adventures of Josie True,” the first net-based adventure game for girls, and is co-director for Rapunsel, a research project to teach girls programming. http://www.maryflanagan.com. Gerngross is a full time writer and game designer with a background in filmmaking and professional software engineering, among other talents. http://www.sharedstate.com.

As system designers, we are concerned with the way computer technology permeates our everyday lives, and how our everyday lives are in turn shaped by the technologies we use. [ineffable] foregrounds digital systems we use everyday in our work and play and examines their complex relationship to language.

Career Moves explores the contradictory world of women in corporate America througan interactive, computer controlled board game.

Work is a troubling condition internationally for all women. Women represent 50 per cent of the world adult population and one-third of the official labour force, but they perform nearly two-thirds of all work hours, receive one-tenth of the world income and own less than 1 per cent of the world’s property. Much women’s work is not officially recognized and does not entitle them to any remuneration, respect or rights usually associated with work (UNESCO).

The game itself represents several aspects of women and work under a variety of conditions, from menial jobs to corporate spaces. Many plastic items are embedded into the game board, representin accoutrements of “success.” Players typically take turns moving their game pieces around the board using the die. Upon landing on a space occupied with an object, players use tongs to retrieve the object. If the player falters or touches the side of the board with the tongs, he or she hears sampled voice sources taken from the career coaching and self-help industry directed at women, groups feeding off of social change by offering patronizing and constrictive advice to women caught in flux. The sampled dialogue presents a tangled web of contradictory statements about women, work, and agency.

The communal aspect of playing the game is intended to be part of the work: because the game appears at first look to be a commercial-style game, players begin in the spot on the board accordingly marked. This commercial style is intended as a critique of the historical sequence to which the popular board game belongs: many games have traditionally supported social “norms,” including heterosexuality, consumerism, and especially non-liberatory positions for women. However, as players progress down the board, it becomes clear that it is they themselves who are determining the rules of the game, and the collective and individual goals become apparent. This not only turns the collective experience and memory of such “family” style activities around, but turns the game experience into a collaborative rewriting of such conventions by calling into question the very motivation behind such goals.

Collective creation, to Levi-Strauss, involves forming commonly understood rule-goverened semantic systems to make objects, especially art objects, accessible to the community. When players begin to define and defend the game as they play it, a dialogue opens about consumption, work, and the political and economic machinery behind the production of a relatively innocent-looking pasttime. Thus the work is best seen in settings, where groups can take turns experiencing and creating their own semantic system.

Physical specifications:
Gameboard is 20″x 30″, controlled by a pc laptop
Game kit includes: two sets of headphones, die, player pieces, corporate objects for removal
Game sits on table, provided, or within the site