|
www.dichtung-digital.com/2005/1/Wardrip-Fruin Playable
Media and Textual Instruments The impetus for coming to this
term was not a love of terminology, but the author’s
need as an artist to situate a set of experiments
in creating "instrumental texts" and "textual
instruments" within an appropriate context. While
it doesn't make sense to discuss all of these
experiments as games, what distinguishes them from
other electronic texts is their playability —
both that they can be usefully considered as
playable, and their particular structures of play.
This essay discusses, particularly, two "textual
instruments" recently constructed by the author in
collaboration with David Durand, Brion Moss, and
Elaine Froehlich. While both of these instruments
operate according to the logic of n-grams (as first
used in textual play by Claude Shannon), one
instrument is designed to play with known local
texts while the other is designed to employ the
contents of network RSS feeds and web pages. One
composition for each of these instruments —
Regime Change and News Reader,
respectively — is considered.
|
|
1.
This is not a game I'm
writing this in the first person. For clarity’s sake.
Because, while I am writing about terminology, the terms I
discuss are simply those that I use to think through my work
as an artist and its relationships with the work of others.
If employed for different purposes, as we say online, “your
mileage may vary.”
The
first term I need to consider is “game”, but
from an oblique direction. Because I need to consider some
things that are “not games.” “This is not a game” was a slogan, at one time, embraced by a group of hypertext writers and theorists. While it is often traced back to a 1988 hypermedia writing workshop run by Rob Swigart, the slogan’s most visible proponent was John McDaid[2]. McDaid may be best known as the author of the “artifactual” fiction Uncle Buddy’s Phantom Funhouse (published by Eastgate in 1993). Other proponents of the slogan included high-profile members of the hypertext community such as Stuart Moulthrop and Michael Joyce.[3] Swigart has written for computer games and published traditional novels — at the time of the workshop he’d recently published the “computer novel” Portal through game developer Activision.[4] And, in fact, we can see the proto-slogan in use in discussions of Portal from before the convening of the workshop: "Emerging
from the mists of the 'Vaporware' list in PC
Letter, Portal has at last been published by
Activision. Having thus established that Portal is
not vapor, its creator, novelist Rob Swigart, has some
further observations about what Portal is and is
not. McDaid writes the following of the workshop group’s enthusiasm for “this is not a game” as a phrase: “We thought this so important that we put it on the t-shirt, in real big letters.” Why was it so important? In part, McDaid tells us, it was practical. There was no way to compete with games, especially graphically (figure 1), so it was better to stay off that turf. But he also writes of what he calls a “larger, fictional issue.” An issue connected to the structure of game play: "The payoff for 'correct' play [is] usually to win; to play 'incorrectly' is to lose. This is very much at odds with what one might loosely call goals of fiction: exploration, insight, and the renewal of the perceived world through alterneity." But McDaid then goes on to say, in the next sentence, “though it is true that in my own fiction, Uncle Buddy's Phantom Funhouse, there is in fact a puzzle.” Faced with this, I’m a bit puzzled myself. Certainly I wouldn’t want to elide the very real structural differences between fictions like McDaid’s and those for which Infocom is best known. At the same time, however, there are also significant differences between McDaid’s and Swigart’s — not the least that one has “no puzzles to solve” while in the other “there is in fact a puzzle.” Why, in these circumstances, would these writers have chosen such a primary focus on the term “game”? When I interviewed Stuart Moulthrop, I asked him about this focus, about the hypertext community’s version of “this is not a game.” He pointed out that the implied comparison between the works on each side of the phrase wasn’t neutral: "There
was an element of rank professional jealousy, for sure.
[Infocom] had a market, after all. We were stuck in
the garage. In retrospect our allergy to games looks
incredibly foolish, both because Infocom's market experience
didn't end all that happily, and more important because
there was so much good work coming out of [the interactive fiction]
community, and still is. Here we see “this is not a game” functioning as a distinction between mere text games and work worthy of consideration by the literary community. Between low and high culture. Between trivial play and serious writing. Interestingly, those on the other end of this distinction seem to have chosen a different approach. As it happens, the piece quoted above discussing Swigart’s Portal was published in Infocom’s house publication, The Status Line (previously The New Zork Times). It concludes: “Can interactive storytelling work without challenging puzzles or conflict resolution? Portal proves it can.” This is not an endorsement of a competitor’s product (Infocom had recently been purchased by Activision) but, unless motivated entirely by command of their new corporate owners, it is a somewhat surprising endorsement of an artistic project that the “other side” came to see as sharply incompatible with Infocom’s. Personally,
I’m interested in the ongoing work of both of these
communities, and I’m looking for terminology that can
help me see the common ground that the authors of The
Status Line recognized. The promotional game for the movie A.I. had no official name, but here I’ll use its nickname: “The Beast”[6]. Just as the game had no official name, it also had no marketing (in fact, was unavailable for purchase), and no official beginning. Or, to put it another way, it began when and how people began to play it. For many it began with the second A.I. trailer, in which “Jeanine Salla” is credited as “Sentient machine therapist”[7]. Players’ web searches for these terms revealed the beginnings of a trail that threaded through texts, images, and movies across the internet — as well as phone calls, faxes, US Postal Service deliveries, bathroom walls, and live events. The game was a huge success — not only in the estimation of its players, but also in the surrounding media attention (which helped generate interest in A.I.). As the hype reached its height, in May 2001, a television commercial for A.I. was released that contained the words “THIS IS NOT / A GAME” (figure 2).
As Jane McGonigal writes: "This message has since become the mantra for both players and developers of immersive entertainment. To 'TING' a game now means to explicitly deny and purposefully obscure its nature as a game, a task that has become increasingly difficult as immersive players grow more savvy about TING techniques."[8] Here “this is not a game” has a different feel to it. Still a slogan of sorts, but a game’s slogan. And as McGonigal points out, TING is now a game design technique — one that helps support players in their performance of belief in the game’s reality, a feature identified as key to enjoyment of the “alternate reality” gaming genre that The Beast helped launch. This denial of the game’s apartness from reality may seem to refuse the “magic circle” that has been part of our discussion of games since Johan Huizinga's Homo Ludens. But rather than an erasure or breakage of the circle, it’s probably more of a smearing or smudging — a deliberate extension of the border between a game and the rest of life, in order to create space for performance and play. “This
is not a game” could also be seen as a more formal
characterization of The Beast. After all, it had no
rules, no points, and only ill-defined outcomes toward which
the players could work (locating what might be a puzzle,
trying to solve it, hoping to find larger patterns leading
toward the solution of the emerging murder mystery). But
more on this sort of consideration is to come.
One source of such arguments is formal game definitions. For example, Rules of Play, a game design text from MIT Press, defines games as “a system in which players engage in an artificial conflict, defined by rules, that results in a quantifiable outcome”[10]. The focus on quantifiable outcomes (which The Sims may not sufficiently possess) makes non-games, or borderline cases, of many experiences that we commonly call games, including simulation games and role-playing games. The closest these experiences get to meeting the definition is when considered in terms of interim quantifiable goals the players set for themselves. As Salen and Zimmerman put it, “As with other open-ended game-like experiences such as Sim City, RPGs have emergent quantifiable goals but usually no single overriding outcome”[11]. And these authors are far from alone in proposing formal game definitions that systems such as The Sims fail to meet. But more “popular” publications also question the status of The Sims as a game. For example, several game publications’ reviews of later installments (console versions of The Sims, or The Sims 2) note this attitude toward the first installment, for example: "With
the first game, people complained that The Sims was
more like a toy than a game."[12] Sometimes the difference is split: "The original Sims was as much a toy as it was a game."[14] Even the Wikipedia entry on The Sims also (as of this writing) notes that “It has been described as more like a toy than a game”[15]. Taking a different tack, some have sought to more clearly describe how The Sims deviates from the usual definitions of games and then describe this as an alteration of their game model (rather than a lack). Jesper Juul, for example, writes: "Open-ended simulation games such as The Sims change the classic game model by removing the goals, or more specifically, by not describing some possible outcomes as better than others."[16] This could result in us changing our game definitions or, as Juul does, classifying The Sims as a borderline game. I don’t
mean, by this, to seem to be arguing against formal game
definitions (or popular perceptions). It may well be that
The Sims is not a game. At this point I’m
simply intending to add “doesn’t meet my
(in)formal definition” to the senses in which people
have meant “this is not a game.”
Mateas and Stern’s project is part of a genre of entertainment/art software that has been called “interactive drama” or “cyberdrama” — popular knowledge of which has been spread by books such as Janet Murray’s Hamlet on the Holodeck[18]. While interactive dramatic experiences may fail to meet many of the criteria of formal game definitions, this isn’t what Mateas and Stern were arguing in their SIGGRAPH slides. Rather, they were seeking to draw attention to the different structures of play, and goals of play, in this genre. Here, there is no winning or losing, and the point of interaction is not to accomplish a game goal. The interactor certainly forms opportunistic short-term goals, but usually in terms of the dramatic situation. In this case, to be “not a game” is to be a different kind of playable digital media, calling for a different type of engagement, than usually comes to mind when the term “game” is used. The assertion that interactive dramas are not games is relatively well accepted among game scholars. But it is not universally accepted, either within or outside such circles. And it seems that one place it is less accepted is within the game development community itself. Façade was chosen as a finalist for the 2004 Independent Game Festival awards, which must on some level be seen as acceptance of its status as a game by the jurors of that competition. Further, the IGF is held in conjunction with the yearly Game Developer’s Conference — and, therefore, in 2004 Façade was on display in the main exhibition hall for the length of the U.S.’s primary gathering of game developers. I spent time at Façade’s kiosk regularly during the conference and observed the people gathered around, watching others interact and waiting for their turn. I didn’t hear a single one say that it was “not a game.” And neither Stern nor Mateas would have uttered the phrase then, either. By 2004 Façade’s development had progressed further, and their presentation of the relationship between Façade and the term “game” had evolved considerably. While “it’s a story, not a game” had helped shock some out of the “how do I win?” mindset, there is much in Façade that is usefully considered with the term “game.” As Stern described it to me in an email: "In fact, one of the underlying interaction mechanics of the first half of Façade we call the 'affinity game', where Grace and Trip interpret everything you do as a zero-sum taking sides 'game'; the second half of Façade we call the 'therapy game', where the player is (purposefully or not) potentially increasing each character’s degree of self-realization about their own problems."[19] Stern and Mateas here are making an interesting maneuver, introducing “game” in the interpersonal sense (as in Games People Play). As Stern explains in a post on the collaborative blog Grand Text Auto titled “Head Games,” this isn’t a way of trying to shift the ground away from discussion of computer games, but to imagine a new area of the computer game field which focuses on interaction with richly realized characters. Stern writes that, in such games: "The
gameplay will literally need to be about the characters
themselves. The 'state space' the player manipulates —
the variables you affect, the values you change — need
to be the feelings, emotions and thoughts of the characters,
not just external counters, scores, levels and objects. Just
as Façade attempts to integrate such games
into an overall dramatic structure, more traditional game
developers could integrate them with mainstream game genres
involving human characters — from role-playing games
such as Knights of the Old Republic to simulation
games such as The Sims. Of course, for this to be
successful, more research of the sort being undertaken by
Mateas and Stern will be necessary. Taking this point of view, I have a series of thoughts: It may be that none of the examples in the preceding section are games, or some of them may be — and that’s fine. Some of what I create as an artist may be games, or not — and that’s fine, too. But thinking about how these may or may not be games has led me to identify something that is interesting about all the preceding examples, and hopefully also about some of what I’m creating as an artist — how they are played. And this is what has led me to talk about “playable media.” For me, this phrase shifts my thinking from a question I’ve found only temporarily useful (“Is this a game?”) to one I have found rewards sustained attention (“How is this played?”). “Playable media” also encompasses a body of work that I want to consider — including the examples above, as well as many other products of the commercial game industry, and also the body of what might be called “playable art.” By “playable art” I primarily mean projects from the digital art community such as Camille Utterback and Romy Achituv’s Text Rain (figure 5), which invite and structure play.
Of course, I could also talk about all of these things by using an already-popular umbrella term such as “interactive.” But these terms are often overly broad even for my purposes, and the impetus that allows them to attain their breadth may shape their meanings in a way that isn’t helpful. For example, while it’s easy to imagine an argument about whether a love letter is more “interactive” than a computer game, it’s unlikely that we would spend long arguing which is more fruitfully considered in terms of its playability. And while playability is a useful way to look at agreed-upon games (such as football) it’s also appropriate when considering certain types of related play (such as hackey-sack) without encompassing too much (it leaves aside sports commentators, the symbolism of team logos and names, the economy of sporting goods, etc.). And a
focus on the playable also attracts me for another reason —
because we play more than games and “not a”
games. We also play instruments, and compositions. And it is
at the juncture of these senses of play — that for
games, and that for music — that a thought-provoking
discussion about playable texts has been taking place in the
electronic literature community. "My point is that we are currently writers trying to build relatively simple textual instruments that are intuitive and, hopefully, both affective and significant when they are played. I mean played as musical instruments or sequencers or mixers are played. This is ergodic indeed, but still distinguishable from (hard) work or from the type of play in games which is rewarded by winning, by other forms of 'success' or simply by 'playability'."[21] With the term “ergodic” Cayley is referencing the work of Espen Aarseth, whose Cybertext: Perspectives on Ergodic Literature[22] is a touchstone in the electronic literature community. Aarseth defines the term by stating, “In ergodic literature, nontrivial effort is required to allow the reader to traverse the text.” More recently Aarseth has become one of the leading figures in the emerging field of game studies, helping found the field’s first journal as well as the Center for Computer Games Research at the IT University of Copenhagen. In referencing Aarseth’s work, Cayley brings to the fore a focus on play as “nontrivial effort” in music, games, and instrumental texts. In my interview with Stuart Moulthrop cited above, I had the opportunity to ask him about instrumental texts. He, too, discussed some of the potential challenges of the reader/player’s nontrivial engagement, but also some of what authors of instrumental texts might learn from the designs of folk instruments: "Maybe some instruments will be hard to play. They may require practice. Or not. As a teacher once said to me about the guitar, 'After five or ten minutes you'll make sounds that are almost musical. That's what the frets are for.' And that's a great virtue of folk instruments. They do allow you to get in touch with a productive vocabulary very quickly. I think a good instrument would do that. It would stimulate engagement. It should make people want to get in there and interact, and to repeat the experience."[23] But for Moulthrop, perhaps unlike Cayley, musical and literary figures are not the only ones brought together in this discussion. For Moulthrop musical figures are a vocabulary that can help one imagine projects that occupy a space between two other types of work at play in discussions of instrumental texts. As Moulthrop puts it: "What I'm particularly taken with is the notion of a middle space between literary texts and ludic texts — between interactive fiction, or hypertext fiction, and games. You have, with instruments, a text with behavior and temporal dimensions that in some ways maps onto the temporal experience and interactive possibilities in game design." In focusing on games, Moulthrop has likely chosen the stronger of the two comparisons. For the examples usually given of instrumental texts, unlike musical instruments, only “play one tune.” Their structures of play and material for play are designed and delivered together, much as those of games. But this is not to say that the musical figure is unimportant. Rather, I think it helps indicate the sort of engagement that authors of instrumental texts hope audiences will have with their work — again, as with Façade and The Sims, something that is not about winning or losing, or perhaps about quantifiable outcomes of any sort. But, unlike Façade and The Sims, this engagement is a physical discipline, sensitive to differences in movement and able to be learned at a muscular level. Cayley’s
interview statement about instrumental texts actually came
in the context of an answer to a question about his piece
riverIsland (figure
6).
The occasion of my interview with Moulthrop was the release
of his piece Pax (figure
7).
Looking at these may help ground our discussion, as well as
help us understand what other work might be brought into
this category. "If texts are laid out in a regular grid, as a table of letters, one table for the source and one table for the target, to morph transliterally from one text (one table of letters) to another, is to work out, letter-by-letter, how the source letters will become the target ones. Assume your alphabet (including 'space' and apostrophe, 28 letters in all) is arranged in a special loop where letters considered to be similar in sound are clustered together. The aim is to work out the shortest distance round the loop (clockwise or anti-clockwise) from each source to each target."[24] Once the movement for each letter is worked out, the text then moves through fourteen steps (the largest number that might be necessary for any one letter — movement to the opposite side of the 28-character loop). Some letters go through many more transitions than others. Changes are “reluctant” at the beginning of the process and then “anxious” for completion at the end — so that both the early and final stages are close to readable texts. A number of Cayley’s pieces, such as his well-known windsound[25], employ transliteral morphing in a manner that is performative on the part of the program. Texts morph into other texts under the gaze of the reader/audience, using the computational capabilities of the computer on which they are displayed. And yet these morphs could be, like most of the graphical morphs we see, pre-rendered and displayed as moving images (without any computation at the time of reading). The only visible loss would be the small changes in timing from reading to reading on the same computer, and the occasionally larger changes when moving from computer to computer. Cayley’s riverIsland[26], on the other hand, is not only performative on the part of the system, but also performative in a manner controlled in part by the reader. One of the types of performance made available to the reader is relatively straightforward: riverIsland is composed of two loops of poems, one horizontal and one vertical, and the reader can use on-screen arrows in order to trigger movement along these loops. When the reader indicates that a move should be made from one poem to another, the appropriate transliteral morph is performed by the computer. There is another type of reader performance in riverIsland, however, that feels quite different to me. And I believe that this is part of what Cayley was getting at in his talk of instruments during his interview with Stefans. In this type of performance, the reader can click and drag on the screen’s vertical and horizontal Quicktime movies. The vertical movie is an “object” movie that graphically transitions between images of paths through the woods. The horizontal movie is a panorama of a riverside scene. A reader experienced with riverIsland can use these movies to navigate to any point within the work’s two loops. A transliteral morph is then performed between the text that was being displayed before the navigation process began (which might, itself, be an in-process morph) and the destination selected by the manipulation of a movie. This creates an experience for which pre-rendered morphs could not effectively substitute — like Cayley’s figure of the sequencer, it harnesses real-time computational processes to create a performance based on high-level user direction that requires knowledge of its materials and control space.
A sequencer might play itself for some time after being given instructions, but a guitar demands interaction for each note sounded. Similarly, Pax is structured for near-continual interaction. The larger area of the piece, on the left, shows characters floating up (in the first half of the piece’s duration) or falling down (in the second). Unless the reader interacts with these characters, almost no text appears. Readers interact by “catching” floating characters with the mouse pointer. Characters can be released by moving the mouse away, or clicked (either by active clicking, or by holding them caught for 20 seconds). Clicking elicits text from that character, which appears in the area on the right (this becomes a scrolling text area once there is enough text to scroll). The fourteen characters float by in different orders, but those recently clicked tend to reappear, making it possible to consistently evoke text from two or three characters as the piece’s time passes. Each reading lasts from noon to midnight (the characters’ time) and is divided into six thematic movements: "Shaken Out of Time," "American Flyers," "Home Land," "Evil Ones," "Falling," and "Total Information." The text elicited from a character is determined, in part, by the number of times the character has been caught and clicked, as well as the current movement of the piece. The character texts evoke two situations: being caught in some version of a terminal at the Dallas airport (shut down for security reasons in an even-more-irrational “war on terror” than that which now grips us) and being caught in the space and structure of Pax itself (naked, floating, caught and prodded by the interactor). While it would be impossible to manipulate the Quicktime movies of riverIsland toward particular effects without relatively strong knowledge of the piece, Pax provides obvious places to click and quickly-understood effects even for the first time reader. However, because of its random elements and the strong impact of time’s passage, it would be more difficult to exactly reproduce the same reading (after learning to play) than with riverIsland. To put it in terms of the musical analogy, Pax may provide frets, but for an instrument that adjusts its tunings over the course of each playing. And, this, again, points to the strength of computer gaming as a figure for understanding instrumental texts. In the gaming context there is nothing surprising about behavior that changes over the course of time. There is also nothing surprising about the skills of physical manipulation and memorization that would be required to elicit particular readings from riverIsland and Pax. And the fact that these “instruments” come packaged with only one composition, from which they cannot be easily decoupled, also makes sense in the context of computer games. And yet they are clearly not games in the manner that play is approached. Perhaps what the musical analogy helps with most is the fact that these projects seek a lyric engagement — not easily formulated in terms of contest or quantifiable outcome.
The piece I propose adopting for these purposes is New Word Order by Sandy Baldwin (as reported in Funkhouser[28]. In this, the second part of Baldwin’s Black Mesa project, poetry is presented mapped onto objects in a simple Half-Life level. The poetry is that of Billy Collins, the first U.S. poet laureate (2001-2003) appointed during the presidency of George W. Bush. Placing Collins’s poetry within Half-Life subjects it to destruction and reconfiguration with an arsenal ranging from automatic weaponry to the famous crowbar (figure 8).
New
Word Order takes an interaction structure invented for
competitive play with quantifiable outcomes — for
gameplay — and repurposes it as play that
recontextualizes and explores the potential of poetic
language. Screen is a collaboration with Andrew McClain, Shawn Greenlee, Robert Coover, Josh Carroll, and Sascha Becker that was created in the Brown University immersive virtual reality chamber (Cave), as part of a research project in spatial hypertext writing[29]. Brown’s VR chamber is similar to the University of Illinois’s CAVE — a virtual environment that shows three-dimension images while allowing users to continue to see their own bodies, and that does not require users to wear encumbering equipment (unlike head-mounted displays, which are essentially blindfolds with televisions inside)[30]. Brown’s Cave is an eight foot cube, missing its top and one side, and its walls and floor are screens. A Projector is pointed at each screen, alternately projecting streams of images meant for the user’s left and right eyes. The user wears shutter glasses that alternately occlude the left and right eyes, in synchronization with the projectors. The result is stereo VR — 3D vision of computer-generated imagery — combined with the physical presence of the people and objects in the Cave. The initial experience of Screen can be disorienting for those familiar with VR. Rather than make the walls “disappear,” we project flat images onto the same plane as the walls, reinforcing their presence. And the images we project are not of colorful shapes, but of white text on a black background. This text at first appears in an introduction that fades in and out on the walls — and then forms three traditional paragraphs, each nearly filling one of the walls. Each of these paragraphs is a character’s moment of memory that gives rise to the virtual experience of touch. Each wall appears, and then is read aloud. After the last has been read there is a pause, and then a word peels from one of the walls, is spoken aloud, and flies toward the reader. If the reader does nothing, the word circles near her. Soon another word peels, and then another, at an increasing pace, flocking around the reader. The reader can intervene in this process by batting at words with her hand. When a word is hit a sound is heard, and the word flies back toward a wall, perhaps breaking apart in the process. If a hit word is the only word off the walls it will return to the space it left empty. However, if more than one word is off the walls then a hit word may return to a different space.
Once the number of words off the walls passes a certain threshold — something which, with the increasing pace of peeling, only very active engagement can long delay — all the remaining words come free of the walls, swirl around the reader, and then collapse into the center of the Cave (figure 9). A final “closing” text is then heard. In addition to creating a new form of bodily interaction with text, Screen creates three reading experiences — beginning with the familiar, stable, page-like text on the walls; followed by the word-by-word reading of peeling and hitting (where attention is focused); and with, simultaneously, more peripheral awareness of the arrangements of flocking words and the new (often neologistic) text being assembled on the walls. Screen was first presented in 2003 as part of the Boston Cyberarts Festival, and in 2004 it was included in “Alt+Ctrl: A festival of independent and alternative games” at University of California, Irvine. Given
its presentation at Alt+Ctrl, we might simply discuss
Screen as a game, rather than with the more unusual
term “instrumental text.” And, in fact, the
final moments of Screen feature a scattering of
words (and parts of words) on the walls — which caused
one young visitor to the Cave to ask, “Is that my
score?” But while the play of Screen is
reminiscent of classic games from Whack-a-Mole to Breakout,
and some players may at moments be driven purely by the
game-like goal of hitting words as quickly as possible,
there is no contest or quantifiable outcome. Even approached
purely physically, without any attention to the linguistic
nature of the words being played, Screen is more
like hackey-sack than soccer/football. And, in my
observations, players don’t approach Screen
without attention to its words as words. Rather, interactors
oscillate between reading and playing, with the objects of
both coming faster and coming apart, until both experiences
can no longer be sustained and the piece ends. As with
riverIsland and Pax, reader/players can
get better at Screen, though the fact that
interactors do not control the ripping of words alters what
is possible via virtuoso performance. Perhaps the most
impressive performance of Screen I have seen is
that of Michelle Higa, who both edited the video
documentation of Screen and played the role of the
interactor within it.[31]
In order to videotape Screen we had to turn off the
flickering alternation between left- and right-eye images.
Higa had become adept enough at the experience of
Screen that she was able to play it relatively
successfully even without stereo cues. In my DAC paper I described textual instruments as follows: "A
textual instrument is a tool for textual performance which
may be used to play a variety of compositions. In this sense
it is evocative of Thalia Field’s figure of the
'language piano'— something that one learns to play,
and which may produce a much wider variety of texts than is
the case for those projects normally discussed as
instrumental texts.
Arteroids is, on the interaction level, simply the repurposing of an existing game — and, in that way, quite similar to projects such as New Word Order. The major difference, in fact, is that while Baldwin’s piece seems created as a context for Collins’s work, Andrews appears to view Arteroids as an instrument for which many texts could be composed (and then used as materials for play). Andrews created a “Word for Weirdos” to allow others to compose for Arteroids and has included texts from others in presentations of the work — such as the texts by Christina McPhee and Helen Thorington included when Arteroids was shown in the “page_space” exhibition[34]. While on some level it may appear that learning to play Arteroids is no different than learning to play Asteroids, one can imagine the desire to create particular linguistic experiences changing this. With each textual composition as a different starting place, even after the Asteroids-derived parts of Arteroids have been mastered there might yet be much to learn about creating an evocative experience from the play with any one text. And
yet I don’t enjoy Arteroids very much, even
when it uses texts by writers whose work I have appreciated
in other contexts, and even after gaining some experience
with playing it. Considering why this is the case has led me
to define further the work I want to pursue in textual
instruments. But when we think of playing with computers, we generally think of graphical experiences, those that follow in the tradition of Spacewar! (figure 11) rather than Adventure. Created on the PDP-1 at MIT in the early years of the 1960s, Spacewar! was the first modern video game. Two players each had a custom-made controller, which they used to control the flight of a virtual spacecraft on the PDP-1’s CRT. The spacecraft were pulled toward the star at the center of the screen by simulated gravity, and could fire projectiles at one another. A spacecraft hit by the central star or a projectile would be damaged. These are still among the central logics of graphical gaming today — the ability to move graphical objects that on some level represent the player, the ability to fire projectiles, a simulation of some form of physics, and “collision detection” when one thing runs into another. These logics aren’t only the basis for play in experiences such as Half-Life, but also (leaving aside projectiles) in pieces such as Text Rain.
We’re accustomed to seeing successful combinations of graphical logics and game rules repackaged repeatedly. Games such as Pac-Man and Tetris have had many authorized and unauthorized versions “skinned” with different surface graphics and different graphical arrangements, but with the essential logics of graphical movement and gameplay preserved. Such combinations, within a larger range of variation, are also the basis for our identifications of game genres such as “side-scrollers” and “first-person shooters.” I bring all this up in order to make a point about the instrumental texts discussed above — about how they are played. While each of these pieces contains a textual component, they are all played along graphical logics. For riverIsland play is primarily through the graphical/physical manipulation of the Quicktime movies; in Pax it is the collision detection as characters are caught and clicked; in New Word Order the movement of the first-person perspective and collision detection (in firing of weapons and use of the crowbar); and in Screen it is the movement of the interactor’s body and the collision detection of hitting words. What these projects do, in each case, is package together logics of graphical play and methods of response with textual and graphical material. What Arteroids does differently is take a set of logics of graphical play and methods of response and then open them to many different sets of textual material. This might be seen as the same as taking the formula of Pac-Man or Tetris and opening it up to many variations in graphical representation. But, for me, it doesn’t feel the same. Somehow it feels arbitrary, no different than if the graphics in Pong or Spacewar! were opened to replacement by arbitrary text. I’ve been thinking about what would feel less arbitrary. It seems to me that, if the same graphical logic can be skinned with many different surface graphics successfully, perhaps those seeking to create textual instruments will need to consider forms of play that proceed via linguistic or textual logics. Before the computer became part of everyday life, textual forms of play such as the crossword (or games of the Surrealists or Oulipo, or Madlibs, and so on) successfully accommodated many different texts by structuring play around the features specific to textuality. Perhaps the true challenge of creating textual instruments involves finding such structures that benefit from the computational environment.[36] If such structures can be found I also suspect that the result will feel, at least to me, in some ways like a deeper engagement with text than is possible withprojects (like Screen) that proceed along graphical logics — projects that could still be played, though perhaps not as rewardingly, if their words were all converted into colorful boxes. Of course, the next question is where one might begin to explore such textual or linguistic logics. Though perhaps this work has already begun, with those creating computer-based versions of crosswords, Surrealist games, Oulipian games, and so on. And Fields of Dream[37], by Nick Montfort and Rachel Stevens, goes further — bringing the basic fill-in-the-blank logic of Madlibs into a project specific to the networked computer. But
there is also another territory to explore. This is that of
linguistic and textual logics previously employed in text
processing and generation — in contexts ranging from
the computer science subfield of natural language processing
(NLP) to the artistic contexts of John Cage, William S.
Burroughs, or Jackson Mac Low. Whether considered scientific
or artistic, these methods have generally been operated in
batch mode — either generating chunks of language or
analyzing chunks of language. But there is nothing to
prevent them from being run interactively, or to prevent the
interaction with them from being structured as play. Shannon’s nickname comes from the fact that he was one of the major figures in formulating the mathematics of communication — what he called “the fundamental problem of . . . reproducing at one point either exactly or approximately a message selected at another point”[38]. This quotation comes from a paper that was a milestone in the field, both when published on its own (as “A Mathematical Theory of Communication”) and when repackaged as a book with an introduction by Warren Weaver (as The Mathematical Theory of Communication). While a milestone, Shannon’s paper certainly doesn’t have to be approached as a millstone. Rather, it can be approached playfully, an approach to the products of science and technology practiced by Shannon himself. (His famous projects of this sort include a mechanical rat for solving mazes, figure 12, as well as a machine that, when turned on, performs only one action: causing a mechanical hand to reach out from a box and turn the machine off.) One aspect of the paper amenable to a playful approach is Shannon’s description of stochastic approximations of English.
Shannon’s discussion includes a number of sample messages. One is purely random. But the others have contents that are shaped by the frequencies of particular letters or words in the English language. This shaping is of two sorts. In one type of shaping, individual letters or words are selected in a manner weighted by their frequency in English. In the second type of shaping, letters or words are selected in a manner shaped by the frequency with which they appear in groupings of letters or words in English. So, for example, “E” is a more common letter than “U” in English. However, if there is a pair of letters that begins with “Q” it is much more likely that the complete pair will be “QU” than “QE.” Taking the frequencies of pairs into account in this manner means paying attention to the frequencies of “digrams.” Paying attention to sets of three is attention to “trigrams.” And, more generally, paying attention to the frequencies of groups of symbols (rather than only individual symbols) is the use of “n-grams.” Shannon provides six sample messages. In the first, each of the 26 letters and the space appear with equal probability: "XFOML RXKHRJFFJUJ ZLPWCFWKCYJ FFJEYVKCQSGHYD QPAAMKBZAACIBZLHJQD." In the next, the symbols appear with frequencies weighted by how commonly they appear in English text (i.e., “E” is more likely than “W”): "OCRO HLI RGWR NMIELWIS EU LL NBNESEBYA TH EEI ALHENHTTPA OOBTTVA NAH BRL." In the third, symbols appear based on the frequencies with which sets of two of the symbols in English. That is to say, after one letter is recorded, the next is chosen in a manner weighted by how commonly different letters follow the just-recorded letter. The sample message created in this way is: "ON IE ANTSOUTINYS ARE T INCTORE ST BE S DEAMY ACHIN D ILONASIVE TUCOOWE AT TEASONARE FUSO TIZIN ANDY TOBE SEACE CTISBE." In the fourth, symbols appear based on the frequencies with which sets of three of the symbols appear in English. This, again, is called a “trigram” — with each choice of the next letter being weighted by the frequencies with which various letters follow the set of two just recorded. This sample message is: "IN NO IST LAT WHEY CRATICT FROURE BIRS GROCID PONDENOME OF DEMONSTURES OF THE REPTAGIN IS REGOACTIONA OF CRE." In the fifth, the unit is moved from letters to words. In this message, words appear in a manner weighted by their frequency in English, but without attention to the prior word: "REPRESENTING AND SPEEDILY IS AN GOOD APT OR COME CAN DIFFERENT NATURAL HERE HE THE A IN CAME THE TO OF TO EXPERT GRAY COME TO FURNISHES THE LINE MESSAGE HAD BE THESE." Finally, in the seventh sample message, words are chosen based on the frequency with which pairs of words appear in English. This, again, like the technique of choosing based on pairs of letters, is called a “digram” technique. The final message is: "THE HEAD AND IN FRONTAL ATTACK ON AN ENGLISH WRITER THAT THE CHARACTER OF THIS POINT IS THEREFORE ANOTHER METHOD FOR THE LETTERS THAT THE TIME OF WHO EVER TOLD THE PROBLEM FOR AN UNEXPECTED." These messages are interesting in part because of how they are generated. The first two were created using a book of random numbers, with the addition of a table of letter frequencies when creating the second. But the rest of the samples were constructed by using ordinary books (making the assumption that ordinary books appropriately reflect the frequencies of letters and words in English). Shannon explains the process as follows: "To construct (3) for example, one opens a book at random and selects a letter at random on the page. This letter is recorded. The book is then opened to another page and one reads until this letter is encountered. The succeeding letter is then recorded. Turning to another page this second letter is searched for and the succeeding letter recorded, etc. A similar process was used for (4), (5) and (6). It would be interesting if further approximations could be constructed, but the labor involved becomes enormous at the next stage." That is to say that the last sample message (which begins with a sequence that sounds surprisingly coherent) was created by opening a book to a random page, writing down a random word, opening the book again, reading until the just-recorded word was found, writing down the following word, opening the book again, reading until that second word is found, writing down the following word, and so on. This is a method that sounds remarkably like some of the kinds of textual play engaged in by experimental artists and writers. It certainly sounds like a technique that could be brought into an explicitly playful context. The surprising coherence of the last sample is also of interest. It shows that processes of the sort being used by Shannon have potential as a form of logic that can be made operational with linguistic material. Of course, the sample is quite rough, in part because only one previous word is being taken into account at any time. What Shannon calls “further approximations” — for example, taking two, three, or more previous words into account — certainly gives more English-like results. Shannon pointed out correctly that, using the method of flipping through a book, the labor involved in creating such further approximations would be enormous. But the modern availability of computing power has made carrying out such calculations automatically a near-trivial task for reasonably-sized bodies of sample text. As Shannon also pointed out, the stochastic processes he described are commonly considered in terms of Markov models. And, interestingly, the first application of Markov models was also linguistic and literary — modeling letter sequences in Pushkin’s poem “Eugene Onegin”[39]. But Shannon was the first to bring this mathematics to bear meaningfully on communication, and also the first to use it to perform text-generation play. This
model (whether called n-grams or Markov chains) is now
widely used in natural language processing and generation,
often in combination with other techniques. It has also been
used in electronic literature, perhaps most extensively by
John Cayley. At least seven of Cayley’s works employ “collocational”
word-level digram procedures, including Book
Unbound[40]
as discussed in Aarseth[41].
Last but not least, this approach has also been the primary
basis of textual toys such as the DOS program
Babble!, the emacs “Dissociated Press”
command, Hugh Kenner and Joseph P. O'Rourke’s
Travesty, Andrew Plotkin’s chan.c,
and Brion Moss’s prate — which have
themselves at times been used in the generation processes
for (non-playable) literature. However, as noted above, both
the elit and toys based on n-grams have operated entirely in
“batch mode.” That is to say, the interactor requests a
body of text, and then that text is produced —
following which the text can be read and another text can be
requested, but no interaction with the texts (or interaction
during generation) is possible. Given this limited nature of
play with n-gram texts, there is also limited context for
play — usually a blank text buffer for the program to
write text into. After talking with Moss (with whom I’d
collaborated on The Impermanence Agent[42])
about these issues, we began to imagine possibilities for
n-gram play that was less batch-oriented and took place
within a textual context. Two major design decisions were made early on. The first was that, rather than build an n-gram text into an empty text buffer, play would always begin within the context of a pre-created document and consist of progressive alteration of that document. This was motivated, in part, by the fact that, while the text produced by n-gram algorithms has microstructures that are recognizable from its source texts, the larger structures of n-gram texts tend to be very similar regardless of the starting material. Some have tried to address this by looking at larger structures in the source texts statistically, but unless the texts in question have been marked up by a human author or editor, this process involves a series of assumptions about the text (e.g., that a period marks that end of a sentence, as it does not in the case of “e.g.”) that are both sometimes inaccurate and on some level aesthetically displeasing. These assumptions are displeasing because they depart from the purity of the simple n-gram algorithm, which in its basic form would work with starting texts in Japanese or Braille or musical notation as easily as English-language ones. However, there was also another motivation. In many n-gram texts, especially those based on short chains, part of the pleasure is based on play between coherence and incoherence — and we found something more interesting, and potentially more meaningful, in such borderline coherence occurring within the context of traditionally-created texts. The second design decision was the identification of our basic method for making n-gram generation playably interactive, rather than oriented toward large batches. We decided that, in addition to the starting document (within which play takes place) we would have a body of text used for producing the alterations to the starting document. (We call this second body of text the “alteration text” or “alteration corpus.”) When the starting document was displayed, certain words would be highlighted. We chose this as a convention familiar from hyperlinks on web pages, letting interactors know that a click will elicit a response. However, these words are not highlighted as the result of author-specified links. Rather, they are highlighted because a string of n-gram text (of a length specified by the piece’s author) appears in both the starting document and the alteration text. We decided that such “bridges” between the two bodies of text would offer interactors the opportunity to open up the starting document and insert text generated from the alteration corpus. More than one generated text would be offered for possible insertion, allowing the interactor to choose one or none (this last leaving the text unaltered). The texts offered would, themselves, be generated from the alteration text through the use of n-gram techniques. The number of texts offered and the n-gram length used in their production would, again, be determined by the piece’s author. Once these decisions were made, we sketched, mocked up, and eventually tried to make operational a number of interaction designs. Some didn’t give the kinds of results we’d hoped for, and others were too computationally expensive to work, but we eventually settled on one that — for us — is satisfying in terms of the feel of interaction and the shape of the attention to text it creates. We’re still discussing some potential variations on the visual/spatial aspects of the interface, but for the first compositions (Regime Change and News Reader) a simple, web-style series of windows seemed both appropriate and easy to implement. More information on the details of interaction will appear below, in the context of the discussions of these two compositions. But first it’s worth noting what differentiates the two instruments we constructed. Our first instrument was the simpler of the two. It depended on both the starting document and alteration text existing on the local drive, with known file names. The second instrument adds a number of features, including the request, processing, and display of network RSS feeds and HTML files. As a result, the first instrument is better suited to compositions involving longer chain lengths (and therefore greater coherence) because it doesn’t have to take time for network file requests or for processing the wide variety of network html files into text that the system can use. The second instrument is, of course, more suited to compositions involving dynamic network data sources — in fact, we regard it as a type of alternative browser. While
some of this discussion may have been a bit difficult to
follow in the abstract, hopefully the following two examples
will make things clearer.
Once the window with Regime Change’s starting text is opened, words in that text, pair by pair, become highlighted. Clicking on words opens a new window (figure 13a). Interacting with new windows produces new texts that will take the place of the clicked words. New windows contain texts that begin with the words clicked in the previous window. Each paragraph in the new window is an alternative text — beginning with the same words but potentially (though not necessarily) following many different paths from there. These texts are generated by connecting chains of words (3-grams and 4-grams) that may have appeared originally in very different parts of the source document. A new window's texts, once displayed, also begin to have words highlighted within them. Clicking highlighted words will open another new window, containing generated texts that can take the place of the clicked words (figures 13b-c). Opening several generations of windows opens wider possibility spaces for the texts that will be created (and that will replace the clicked words in earlier-generation windows). Windows alternate between generation from the Warren Commission text and the original news story.
In any window with generated text, clicking a non-highlighted word is also a means of interaction. Such a click will close the window — and select a text. The selected text will run from the beginning of the clicked paragraph to the clicked word. That selected text will then take the place of the words clicked to open the window (figure 14a-b). This creates a kind of stretchtext — the pair of words clicked to open a window are replaced with the words selected in the open window (usually more than a pair). After opening several layers of windows, part of play is keeping track of where each window came from — so that it can be collapsed by selecting a word that will make a pleasing segue at the point where it will join the text to which the player intends to connect it. (This may be more than one layer down.) Keeping track of context is made easier by the title bar of each generated window — which displays the two words that will be replaced by the generated text, followed by the two words that appear after them in the text clicked. I find that, when I’m playing, this cycle of activities — reading, remembering context, selecting a place to click, reading again — consumes my entire attention. I’ve found it impossible to “give a reading” of Regime Change as I might with other writing projects. My most successful presentation so far, instead of a traditional reading, was a performance in which I played the text and Popahna Brandes read the results aloud.
Most of the interactions I’ve described are those that I would consider part of the instrument. I consider the composition to consist of the selection of texts, the basic settings made for the instrument (n-gram chain lengths, number of alternative texts generated, various selections of colors and fonts, and the ways that different windows open into different texts), as well as the ways that settings change over the course of interaction with the piece (including a “surprise” third text that becomes part of Regime Change’s material after a certain number of third-level windows have been opened). Of
course, I’m aware that this particular set of terms is
not the only reasonable way to understand Regime
Change. Rather than a focus on concepts such as
instrument, composition, and play, it would also be
reasonable to view the piece in terms of something like Nick
Montfort’s “human-computer co-authorship”[44].
Such a focus would seek to make more explicit the moves made
by system and interactor during the creation of the final
textual output from a session with Regime Change.
Employing Montfort’s framework, we would say that the
initial move is: Computer – G (the computer
provides the initial text). Then: Interactor –
I (the interactor provides “some instructions or
intermediate text” by clicking). Followed by
Computer – G (the computer generates texts
based on where the click was placed). Finally,
Interactor – I, which can lead to A by
extension or Computer — G (the interactor clicks,
which may result in alteration of text or the generation of
further text). Then these final stages are repeated,
indefinitely, until the concluding text is reached. This
enumeration may seem an odd exercise, but it does help us
formalize how the texts of Regime Change differ
from those created in other situations of human-computer
co-authorship. For example, while human participants
certainly shape the text created during interaction with
Regime Change, Montfort´s model makes it clear
that interactors at no point generate text. When News Reader launches it displays a window containing the current headlines from the Yahoo! News RSS feed (figure 15). Clicking the headline or preview text opens another News Reader window, displaying the story.
A link below the text preview copies the address of the story to the system clipboard, so that it can then (if this is desired) be opened in a traditional browser of the reader/player's choice (figure 16).
If a
story is displayed in a News Reader window, links
appear within it (figure
17a).
As with Regime Change, these links don't lead to
other web pages, but rather generate texts out of a
statistical text model (in this case, trigrams of the
alteration corpus and their relative frequency). These
generated texts appear in a new News Reader window
(figure
17b).
The alteration corpus is created from the texts of
alternative news stories (found at Common Dreams)
downloaded in the background when News Reader is
launched. As with Regime Change, windows of
generated text contain several paragraphs, each of which is
a continuation of an n-gram that begins with the words in
the clicked window just prior to that word clicked. And,
again as with Regime Change, clicking a
non-highlighted word will close a window of generated text,
replacing the words clicked to open that window
(figure
17c).
The words used to perform this replacement will be those
between the clicked word and the opening of the alternate
text (“paragraph”) it was within. Eskelinen observes that this fear of variety, coupled with our need to grapple with digital objects that produce great variety, may be part of the attraction of computer game studies for some scholars. In computer games variation is pacified by rules and goals. Expanding this, Eskelinen writes that when we have lost “the safe and somehow manageable totality, be it coherent or not” we reach a point where “computer games are interesting, as they domesticate the excess looming large in both ordinary and avant-garde products and processes, and the fundamental potential for change and unreliability inherent in new media objects. ”Perhaps this is also the attraction of performance, of instruments, as a way of talking about digital objects that produce great variety. Performance, and especially improvisational performance, is different each time — and yet we understand that it is structured. Perhaps this is also the attraction of discussing work in terms of playability — in terms of the potential of, and structures for, play. In a different section of the essay Eskelinen puts his finger directly on the challenges facing those interested in creating playable texts: "In any case an instrument is supposed to shape and frame the player’s action and to produce interesting variation. This is a challenge that goes far beyond the overly hyped problems of non-linear presentation. As in any economy of means and ends, it is important to find suitable goals and patterns of change and variation in the functional and causal framework." It is my hope that Regime Change and News Reader on some level accomplish this — providing suitable goals in the production of interesting textual experience, and simple patterns of change and variation based on n-gram logic and interactor selection. It is my hope that they can serve as early steps toward the development of a vibrant area of playable textual experiences, operating along logics more linguistic than graphical. ________________________________________________ |
