02.03.12
Animate Together! MotionPhone app released for the iPad
no comments

MotionPhone was released last week as an app for the iPad. The app lets people create abstract animations together over a network, choosing colors, shapes, and forms that follow their fingers’ most subtle movements. By moving again-and-again in the same animated canvas, people can create layers and rhythms of abstract form and color, much like musicians layer track after track of audio in the studio sessions.

With MotionPhone, people can connect to each other using their iPads for a visual “conversation” using Apple’s Game Center social gaming network. Ordinarily used for leaderboards and multiplayer games, we’ve used Apple’s gaming network for a social form of creativity that updates parlor games like Consequences and Exquisite Corpse into the era of digital animation. And MotionPhone’s iPad canvas is not merely a window, but an infinite two-dimensional world, where people can move about and have different visual conversations at different locations and scales as they pinch in or zoom out.


MotionPhone for the iPad

The original Motion Phone was one of the first networked works of art, and the first interactive networked animation system. Motion Phone began more than 23 years ago, when I was exploring how to make animation directly with my body. In film school at the Rhode Island School of Design, I was inspired by two experimental animation pioneers. The first, Oskar Fischinger, pioneered a cinema of pure abstraction. His earliest films are simple black and white forms, drawn frame-by-frame in charcoal. Yet the resulting movements, such as in Study Number 8 (1931), have incredible emotional power. The second pioneer, Len Lye, pioneered “direct cinema,” created by marking directly on the film surface with pens, inks, or by scratching emulsion off of black leader, as in his masterpiece Free Radicals (1957). These and other film artists’ work are sometimes referred to as Visual Music.


Oskar Fischinger’s Study Number 8, 1931. Courtesy Center For Visual Music. (c) Fischinger Trust 2012

Len Lye’s Free Radicals, 1958

In my enthusiasm for abstract animation, and for interactive computer graphics, I was searching for a way to improvise hard-edged abstraction like Fischinger’s by using my body directly the way Lye did, but with a computer rather than film. In an epiphany one evening staring at the computer, I realized that the cursor was the most interesting object on the screen. Here was the only place that my body, through the mouse, came into the machine. Based on this understanding, I created Motion Sketch, originally written for a Sun SPARCstation, which attaches the movements of one’s hand to the movements of abstract forms. These forms are laid down into a short one-second graphics loop whose temporal complexity results from the continuous layering of forms, creating a rich motion painting.

Motion Phone (1995) was a networked version of Motion Sketch, written for Silicon Graphics workstations. With this software, multiple computers, networked via the Internet, communicate together simultaneously through a shared animated canvas.

As an experiment in abstract visual communication, Motion Phone created new social rules. Instead of a fixed size canvas, Motion Phone provided an infinitely zoomable plane. Multiple “conversations” take place at any position or scale within this virtual world. However, each performer cannot be certain what the other is looking at.

Similarly, performers have independent control of frame rate. If one person sets their frame rate low, they can seem to run circles around the other, creating at miraculous speed and with incredible temporal precision.

Each performer can only erase his own work, and one can also choose to not display anyone else’s work if privacy is desired. However, the background color is set to the last person’s choice so that there is at least one thing to fight over: changes to background color are instantaneous and startling.

By working with a loop, the problem of communication latency becomes irrelevant. As the one-second loop continues over-and-over for each viewer, new material from other viewers appears whenever it arrives. As long as the strokes are revealed within a few seconds of their creation, all the creators have the sense of the lively progress within a shared dynamic canvas.

Motion Phone was first exhibited in 1995 at SIGGRAPH Los Angeles, in a fairly amateur presentation to see how people would respond (see the video above). At the show, I met many contemporaries in abstract animation and interactivity. Computer animation pioneer Larry Cuba spent hours at the SIGGRAPH installation, and introduced me to Elfriede Fischinger, Oskar Fischinger’s widow, whom I had a wonderful afternoon with together, listening to her reminisce about Oskar, and viewing her library, paintings, and notes. Larry also introduced me to Bill Moritz, the foremost scholar of abstract animation, who  had me over his house for a long discussion and tour of his archive, where he gave me some rare Fischinger publications.

Motion Phone won a Prix Ars Electronica prize in 1996, a prize often referred to as the ‘Academy Awards’ of interactive art, and was invited to many international shows afterwards.

Several contemporary digital artists were inspired by Motion Phone, including Golan Levin, whom I showed the program to in 1996 while working together at Interval Research. The experience convinced him that he should learn how to program and make similar works, and we can all be grateful that he did, as he has become one of the foremost interactive artists and educators. When Golan later went to MIT he showed Motion Phone to Casey Reas and others in John Maeda’s legendary Aesthetics and Computation Group, where the beginner’s graphical language Processing was born. My colleague Lukas Girling whom I also met at Interval Research, was also excited by Motion Phone (as was I by his work in interactive music), and we began a collaboration that continues to this day with projects like OscilloScoop.

My hope is that MotionPhone and programs like it can serve as an alternative to ordinary communication like email, text, and video chat—a form of communication where people work together in a meditative, creative process to make something together, and where they experience the kind of concentrated attention that is familiar to artists, musicians, and meditators. This type of attention is different from the neurotic attention of video games, or the sedative concentration of television. Might it even be possible for this social form of networked animation art to become a mass medium?

Read More

Visit MotionPhone at Scott Snibbe Studio for news about the app, and the technical paper Interactive Dynamic Abstraction from the Proceedings of the Symposium on Nonphotorealistic Animation and Rendering, June 2000, describes technical and creative details.

05.03.11
Music of Chance
5 comments

It was 1989 and I was roundly ignoring a verdant spring day in the HVAC hermitage of Brown University’s Computer Science Lab, when Henry Kaufman yanked me up from my workstation to say I must see Merce Cunningham’s dance company perform. Who’s that? I blurted to my friend as he pulled me out the lab’s card-keyed door.

The afternoon rehearsal was taking place in a dimly lit hall, free to anyone who happened to know of it, which turned out to be only us and two or three other students. The dancers on-stage sketched coordinated forms accompanied by eerie rhythmic tones. It was the first time I’d seen bodies lose their individuality—and even their corporeality—to become part of a greater abstraction. Figures hurtled across the stage in a blur; bowed bodies nearly smashed into walls; and dancer’s frames hardened rod straight, then pogoed in syncopation. The movements stirred memories of the abstract films of Oskar Fischinger and Len Lye that I had recently fallen in love with, studying experimental animation at the Rhode Island School of Design.

Henry and I, enthralled, had dozens of questions after the rehearsal. Walking down the dark lanes to the stage, we found a man in the music pit with tussled dark hair streaked with grey filaments. As he disorganized a tattered sheaf of papers, we peppered him with questions, particularly about the dancers nearly hitting the walls, and he explained that the choreography was randomized: stochastic methods gave the dancers freedom to improvise, but also caused close calls. He spoke in great detail about the dancers and their relationship to the music, which he described as contrapunctal: not meant to mirror the dancers, but to enhance and amplify them, like a human relationship.

As Henry and I walked out, we passed a student who asked us eagerly, “What did he say to you?” “Who, that guy from the crew?” “That guy? That guy is John Cage.”

The idea of music and dance that adapted fresh to each setting of stage and audience charmed me. I had been playing with interactive sound and image on the computer since I started programming an Apple II+ as a little boy in 1980 and continued similar experiments into college in the late 80s, even as I labored under a heavy Computer Science course load. Sneaking audiovisual experiments between classwork during late evenings when research computers were idle, the screens came alive with my abstract visual experiments like Motion Phone.

Moving images crave sound, but in trying to add generative music to interactive animation, I continuously ran up against a problem: the music felt annoyingly literal when directly and repetitively tied to the graphics. My most satisfying musical experiments came from the opposite direction: taking a musical track and improvise animation to it. I made a few amateur films improvising to tracks from Mingus Ah Hum, an album I was addicted to to at the time. And I fantasized about working with a modern Mingus like John Zorn; but the impossible logistics of thirty-thousand-dollar computers had me stumped at the time.

In 1995 in Los Angeles, I showed Motion Phone for the first time publicly in SIGGRAPH’s “Interactive Communities,” an exhibit of experimental interactive technologies. It was there that I met Larry Cuba, an abstract animator who refused to leave Motion Phone’s workstations, creating exquisite visual poetry in several hours’ of sessions as SIGGRAPH visitors impatiently queued in a growing line behind him. At the end of one of these sessions he turned to me and said that I must meet William Moritz, the chronicler of Oskar Fischinger and the foremost scholar of abstract animation who lived less than an hour away.

Moritz’s hillside home overflowed with file cabinets and abstract art lit by streaks of California sun shooting through its windows. After a few hours’ note-taking, archive digging, gossip, and history, I found the courage to ask Moritz about my nagging problem with the automated music that often accompanied animation and light performances. He replied: “It is precisely those aspects of music that can be mechanically translated from the visual to aural, which are the least interesting.”

Moritz’s comment crystallized the problems I’d had tying images to music. I started to daydream about an organic, generative way of creating music, similar to Motion Phone, but it was only at Interval Research in 1997, that these ideas found fruit when I met Lukas Girling. He was a new graduate from the Royal College of Art’s Interaction Design Program who had come to California to work with Joy Mountford and Bob Adams in an interactive music research group. I volunteered immediately to collaborate with him, and we combined our complementary interests in capturing the body’s gestures to translate them to image and sound. Using the body to introduce “randomness” through its infinitely varying gestures became the way out of the sound/image conundrum: just as with a musical instrument, refined human gestures provided infinite variation, subtlety, and novelty.

Working together, Lukas and I created several prototype interactive music “instruments” that used the language of DJs to create music in lieu of the traditional score/performance model. Lukas opened up my mind to real-time methods for creating music with the body’s most subtle movements that didn’t require a traditional music theory education: instruments that used body language borrowed from DJ’s fingertips and palms sliding across records and mixers. We had a fruitful collaboration on projects that our small audience adored; and we came close a few times to deals with major video game companies, but unfortunately our work was never publicly released.

One of the highlights of that period was meeting Brian Eno, who visited us to comment on our work. He had a chance to use the three (non-musical) pieces of the Dynamic Systems Series: Gravilux, Bubble Harp, and Antograph, and he pointed out the similarity between his Tape Loop Experiments and Bubble Harp, each of which create an extremely long performance due to the varying durations of each segment: in Eno’s case, tape loops; in mine looping motion gestures. Eno also advised us on the subtleties of tuning old analog synthesizers that later turned into a quantizing feature for our apps. I think Eno also found inspiration in our work for some of the later experiments in audiovisual synchronicity that he’s created in galleries, performances, and later the iPhone app Bloom created in collaboration with Peter Chilvers. Eno’s 1999 article for Wired Magazine The Revenge of the Intuitive written soon after our meetings together at Interval includes many insights into interactive performance.

With these influences in mind, I recently revisited interactive music. Returning to principles from more than a decade before, I re-imagined a way to create infinitely varying music from the formerly silent, and confusingly-named Bubble Harp. The obvious idea that had been with me since the nineties was to pluck each line of the Bubble Harp according to its length. The beauty of this model is that, just like its animation, and true to Cage’s inspiration, the composition will never repeat. Each point, replaying a person’s gesture according to its own duration, dances against the others’ rhythms to create slightly different geometry and sound each time through. At the same time, the variations are constrained, so that the animated drawing becomes a recognizable improvisational structure, like a jazz session.

To make the sometimes dissonant results more musical, I constrained the compositions to specific scales, with the pleasingly pentatonic as a default, allowing creators to tip-toe into music theory as they change into the exotic-sounding Hungarian scale, a familiar Blues scale, and then the full C Major scale to explore music’s full complexity.

Encouraged by Bubble Harp’s success, Lukas and I reunited to collaborate on an instrument inspired by the nontraditional ways of making music that DJs and electronic musicians embrace. In OscilloScoop, Lukas and I have curled the Cartesian X-Y grid of music software back into loops that turn like records, yet sculpt like clay. Among Lukas’ many insights is to turn the relatively impenetrable world of music software such as the ReBirth, Lemur, Pro Tools, Ableton Live, and other hard-core musician’s tools, into something as effortless (and fun) as Super Mario. At Interval we had used video game controllers as the low-barrier gateway to control music, but now with the iPad, we are able to be entirely intuitive, touching music with our bare hands.

OscilloScoop’s logic isn’t a musical innovation: at its core it’s a synthesizer with the three spinning crowns that control pitch, filter, and volume. What is exciting is turning the grid-authoring experience into an improvisational game. Just as Cage used games like the I Ching to create his music, or launched musicians and dancers into action with a small set of rules, we have made a set of rules that inspire improvisation and allow untrained musicians to feel their way through sonic textures.

Some of the world’s great musicians don’t read music, but feel their way through innate improvisations, or cut and paste with software, building up songs micro-slice by micro-slice. With apps like Bubble Harp, OscilloScoop, Thicket, Soundrop, Singing Fingers, and SoundyThingie come new ways to create music for the ordinary person that don’t merely turn you into a Guitar Hero puppet, but allow one to create personal, original compositions from infinite sonic possibilities.

07.30.10
Dreams of Abstract Mass Media
2 comments

Interactive Art’s Entrepreneurial Urge

In 1935 Marcel Duchamp nervously awaited the public in Alley F, Booth Number 147 of Paris’ 33rd Concours Lepin. This exhibition offered inventors a chance to showcase a new product before the public and financial backers. Over the years many notable inventions debuted at the Concours including the ballpoint pen, the steam iron, and the contact lens. However, on this day Duchamp offered a series of abstract interactive artworks called Rotoreliefs, which he had just patented.

Marcel Duchamp's Rotorelief, 1935

Duchamp’s invention was a series of circular disks printed with abstract patterns. When placed upon a spinning record player, the discs created optical illusions of depth, color, and movement. Duchamp won an honorable mention in the Industrial Art category for his invention, which he also mischievously referred to as silent music. Unfortunately, Duchamp’s investment was a failure. He received few orders and was ignored in favor of more practical inventions like those of his neighbors: a pre-Cuisinart food processor and an early trash compactor. Most of his first run of 500 units were lost or destroyed.

Rarely have painters and sculptors attempted to mass-market their works of art the way musicians and filmmakers do. What is it about the genre of interactive and time-based media that brings out the entrepreneurial urge? I believe it is the inherently personal nature of interactivity. An experience like Duchamp’s Rotoreliefs is best encountered privately in an intimate space. Over the course of weeks and years the viewer becomes increasingly familiar with its time-varying and reactive properties, getting to know it like a favorite song. More fundamentally, we recognize that artistic interactivity is modeled on intimate face-to-face interactions with other human beings, in constrast with the less intimate “broadcast” experiences of a gallery, performance, or exhibition.

Duchamp was not alone in his dream of reaching out directly to his “customers.” Now nearly forgotten, but groundbreaking in his time, Thomas Wilfred pioneered light performances decades before their rise in the 1960s. His performances, called Lumia, were created with his Clavilux, a color organ of his own invention. These performances, which took place mostly in the 1920s, were infrequent, expensive, and ephemeral. Seeking to broaden his audience, in 1930 Wilfred premiered sixteen Home Clavilux devices, also known as Clavilux Juniors.

Thomas Wilfred's Home Clavilux, 1930

These devices imagined a world where families would sit down after dinner, open a handsome hinged oak cabinet, and enjoy an evening of time-varying abstract light. The “visual music” inside the box was created by lights modulated by colored disks that reflected and refracted across miniature mirrors onto a translucent screen. In a gesture anticipating John Cage, Wilfred named each Home Clavilux numerically, based on the amount of time before the performance began to repeat: one device was subtitled 8 Hours, 15 Minutes, 42 Seconds. Each Clavilux also included interactive controls to modify the tempo, shutter, and color of the pre-programmed performance. Unfortunately, these devices, like Duchamp’s Rotoreliefs, proved commercial failures, and none were constructed after the initial run. Wilfred almost got it right—people did want to watch a cabinet of light—but preferred the greyscale forms of Jackie Gleason and Lucille Ball to waves of color. Most of the original Home Claviluxes have been lost or destroyed.

Oskar Fischinger's Lumigraph, 1950s

Oskar Fischinger, the master of abstraction animation, also flirted with distributing interactive abstraction. While living in Los Angeles during the latter years of his life, Fischinger invented a “real-time” method to produce abstract imagery. His device, which he named the Lumigraph, consisted of a flexible white screen that he pressed into with his hands, intersecting sheets of colored light controlled via interactive dials and pulleys. Fischinger made several performances over the years with the Lumigraph, and a patent issued in 1955 attests to his hopes to commercialize and distribute the device. However, his dreams were never realized. The broadest distribution the Lumigraph achieved was via Hollywood in the role of a futuristic erotic stimulator in the 1965 film Time Travelers.

Lumigraph Patent, 1955

Lumigraph Patent, 1955

Today we have more hope. With the introduction of the iPad and its smaller mobile cousins, the problem of a device and distribution is solved: interactive artists can now focus on the software “scores” for our intimate, interactive performances. These highly useful devices can be exploited for useless purposes. I feared for several years that my career would go the way of the ignored precursors I revered like Wilfred and Fischinger. In 2002 I stopped creating abstract screen-based interactivity after a dozen years of showing in galleries and museums without sales or wider distribution. Now, after two months in the app store, I have an audience of hundreds of thousands that grows daily. Gravilux, Bubble Harp, and Antograph are enjoyed by people all over the world, bringing the unique joys of abstract interaction to anyone with a mobile device, and taking their minds away momentarily from more neurotic activities. I’m particularly looking forward to seeing modern masters of interactive abstraction like John Simon Jr. and Camille Utterback turn their efforts to this small screen and delight viewers worldwide. Could abstract interactive art become as widespread, and easily “consumed” as music and movies? Stay tuned.

REFERENCES

H.P. Roché, “Souvenirs of Marcel Duchamp,” in Lebel, Marcel Duchamp, 79 and 86. Originally published in the Nouvelle Revue Française, June 1953, 79-87

Thomas Wilfred: A Retrospective. Concoran Gallery Exhibition Catalog. Washington, 1972.

Fischinger, Elfriede, “Writing Light,” reprinted in First Light, Robert A. Haller, ed. New York: Anthology Film Archives, 1998: 30-34. http://www.centerforvisualmusic.org/WritingLight.htm

Personal conversations with Elfriede Fischinger and William Moritz, August, 1995.

Thanks to Cindy Keefer of the Center for Visual Music (http://centerforvisualmusic.org) for up-to-date references on Fischinger’s Lumigraph.