mary
flanagan + andrew gerngross
<read a 2005 review of this project
on neural.it>
We use text so frequently in digital communication. Most technology
users correspond in written word more frequently than in person conversation
or telephone chats. However, we seldom stop to consider “voice”
within our mundane, written correspondences.
[ineffable] is a computer application which reads aggregated emails
between two correspondents and maps the use of language through the
words users utilize in everyday correspondence. The project explores
the question, Do we have particular “voice” in our daily
writing to friends and colleagues, and does that voice change depending
on who we are writing to and why?
Collaborators Mary Flanagan and Andrew Gerngross work to bring into
the foreground the primary form of exchange practiced by many technology
users -- email -- through closely examining the use of language in
email systems. Email is used for work and play, intimate exchange
and legal agreements. How are different kinds of language, and thus
sounds, used in correspondences with different people? How do we "sound"
to those reading our emails, and how does the email of others sound
to us?
Words, phrases, and sentences represent a time, a person, a map of
interpersonal experiences (the external world) as well the way a user
relates to the context of digital communication -- and relationship
of that person to their own computer. This project maps the geography
of these relationships with sound and image.
The [ineffable] program explores the relationship between real people
through their written messages to one another, but further, the work
generates a set of procedures which expose how our everyday experiences
with email encode the kinds of sounds we make, our "voices,"
into our digital systems.
HOW
[ineffable] WORKS.
[ineffable] collects chronological information, time between emails,
length of correspondence, and most importantly, the kinds of phonetic
sounds used by a correspondent in his or her writing and generates
a sonification and visualization of this content. Written in Java,
[ineffable] analyzes a user's emails to or from a particular person
and maps the language used by examining the phonemic makeup of the
words utilized in the correspondence. The work "reads"
a pair of user's email archives and, side by side, analyzes the
words therein, grouping them based on the recipient, date, time
elapse between correspondences, overall amount of correspondence.
Most importantly, we have the program find a "sound signature"
to the words used in words, paragraphs, sentences and finally, the
overall email set.
[ineffable] functions as an experimental system which considers
the “sound in the head” we create while reading and
writing as a synaesthetic experience. The visitor to the work encounters
a set of dynamic sonifications and visualizations from each set
of correspondences in a split stereo sound environment.
Visualizations. The two sides of the visual aspect of the project
represent the application's reading of the email aggregate. Visual
modes switch between a moving line map and a "voice organism"
color collage generated by the system. The moving line map illustrates
sound structures encountered by the system. It shows the frequency
of the phonemes created by the email in its thickness and color.
The voice organism visualizes time and the change in voice through
time. It increases and decreases in complexity and transparency
depending on the dates in between correspondences. The smoothness
is the "consistent" state of the "voice" or
the general phonemic sounds used by the system. Changes in the sound
"voice" change the organism. The system generates "ideal
words" that are the aggregate of a user's most frequently used
sounds. The most commonly used sounds --for example, if a writer
uses many "mmm"s and "er", generates a representative
word from the sounds (in this example, the word could be "murmur").
These are displayed under the visualization and change over time.
Sonifications. In the audio aspect of the work, several audio
tracks are created in response to the email data. First, phoneme
sounds are assigned to an “instrument” created in the
programming API Jsyn. Phonemes sampled from human voices are also
played back in the rhythm of the email’s syllabic structure.
PRIOR EXHIBITION OF THIS WORK.
[ineffable] is a new work and was selected by an international jury
to premier at the prestigious SIGGRAPH computer graphics conference
in the Art Gallery in August 2004.
INSTALLATION DIAGRAM.
This work can utilize a monitor or video projector in its
installed form. It runs off of one laptop, with a mixer and
4 speakers (and optional headphones).
ABOUT THE ARTISTS.
The project is a collaboration between New York artist-scientists
Mary Flanagan and Andrew Gerngross. Flanagan’s computer and
net art has been shown at the Whitney Biennial (2002), the Guggenheim
Museum (2004), SIGGRAPH, and Ars Electronica. Her books include
Reload: Rethinking Women + Cyberculture (MIT Press 2002) and reskin
(forthcoming). She is the creator of “The Adventures of Josie
True,” the first net-based adventure game for girls, and is
co-director for Rapunsel, a research project to teach girls programming.
http://www.maryflanagan.com. Gerngross is a full time writer and
game designer with a background in filmmaking and professional software
engineering, among other talents. http://www.sharedstate.com.
As system designers, we are concerned with the way computer technology
permeates our everyday lives, and how our everyday lives are in
turn shaped by the technologies we use. [ineffable] foregrounds
digital systems we use everyday in our work and play and examines
their complex relationship to language.
|
<<listen
to [ineffable] reading andy's email>>
<<see a
segment of ineffable in action; each of the visualization streams
(left and right side) represents each of the correspondents>>
|
