Studio 99 Melds Art and Science
Microsoft Research
December 3, 2013 10:00 AM PT

When one thinks of corporate art, a large canvas hanging in a lobby usually comes to mind. But In Building 99 on Microsoft’s Redmond, Wash., campus, a gallery space called Studio 99 presents art that melds data, creative coding, and leading-edge technology.

This isn’t too surprising, given that Building 99 houses the headquarters of Microsoft Research. What might be surprising is how researchers view the synergies between art and science.

“We feel the artistic process bears a strong kinship to research processes,” says Asta Roseway, principal research designer. “By introducing a venue for art, we encourage creativity in research, as well as synergies and cross-pollination between the arts and science. The arts offer a way to think differently and challenge assumptions, both of which are critical for a successful research organization.”

“We discussed how the research community could embrace and incorporate art into its process,” says researcher Neel Joshi, “and how increased exposure to art and artists might encourage researchers to explore unconventional paths. We wanted intellectual, visceral experiences that would bring a sense of fun and wonder to the building, with exhibits that walked the line between traditional art and more experimental work.”

Studio 99 opened in November 2012 with exhibits featuring selected works from researchers at the Redmond lab. But even then, an artist-in-residence (AiR) program was part of the plan, and in August 2013, James George was invited to join Studio 99 for a three-month collaboration.

James George
James George

George is a computational video artist and an adjunct faculty member in the Interactive Telecommunications Program at New York University, where he lectures on computational processes in video art. He has exhibited in Europe, South America, Japan, and New York.

“I had been using freenect open-source drivers for Kinect for Windows, reappropriating the camera’s use in image making to explore new forms of cinema,” he says. “There was so much synergy between what I was doing and the work at the lab that the AiR opportunity seemed like a natural fit.”

George joined the Natural Interaction Research group, with principal researcher Andy Wilson as his adviser. During a three-month residency, George worked on three projects.

Grip

He spent his first two weeks in Redmond meeting researchers and learning about their work. As soon as he met senior researchers Charles Loop and Cha Zhang of the Multimedia, Interaction, and Communication group, George knew that the team’s real-time 3-D scene capture and reconstruction technology would complement a concept he wanted to explore.

Grip imagery
Grip imagery

“At the time, their system was set up for head and torso shots,” George recalls. “I wanted something four times larger so that I could film entire bodies in motion. Research Media Works, Microsoft Research’s in-house video-production group, assisted me in setting up the shoot for what would become the Grip video, while Charles and Cha adapted the system to support the installation.”

Grip is a site-specific video installation designed for two-column video displays in Studio 99’s gallery. Each column face acts as a window into an abstract 3-D scene occupied by the figures of two people suspended in a mutually sustaining pose. When a viewer approaches the column, the couple responds by breaking their grip and quickly falling out of view. Then the couple repositions in a different pose on the column, and the cycle of mutual dependency continues. The figures, whose movements are choreographed by George’s collaborator Alice Gosti, indicate how the depiction of the human form can be altered by technological progress.

Grip also explores the concept of inevitability, a concept that has become extremely important to George.

“I’m very interested in technological determinism,” George explains, “in the notion that once a technology is suggested, it will inevitably become part of our culture. Here at the lab, researchers are deciding and shaping the way technology will make its way into the real world. As an artist, I respond to the emotional content of these developments. The concept behind the video shoot was to film people who had just let go of a mutually sustaining pose, so they are falling. There’s a lot of imagery of people falling down. I think this captures a visceral feeling of irreversible action that will get people to think about the way their work might reshape society.”

Wall Queries

Wall Queries visualization
Wall Queries visualization

With the Wall Queries project, George used Internet image-search data to create architectural-scale murals. He views search as a strange middle ground where results can seem totally random, yet are filtered, controlled, and understood through a search engine’s algorithms. Search results depend on the interaction between inadvertent contributions from Internet posts and the programmers’ inferences. Wall Queries incorporates 10,000 images from a single query that specifies color and geometry. The images are arranged intentionally to present a chaotic texture from a distance, but a closer examination makes it clear that each image has a specific reason for appearing in this query.

“It’s one example of how new ways of collecting and capturing visual information have become part of our daily experience,” he says, “and I feel an urgency to reflect on the biases embedded within the formats and practices we are adopting so swiftly.”

George met several times with the Bing team and received help with image search for Wall Queries. He even was invited to attend product meetings to discuss how Bing tools could work better for artists.

Kinect Common Bridge

One of George’s ongoing goals is to find ways to interact more closely with the tools he and other artists use in the creative coding world. During his time at Microsoft Research, he was able to work directly with the Kinect for Windows and Microsoft Open Technologies team to develop Kinect Common Bridge, an interface layer that sits atop the Kinect for Windows SDK that makes the software-development kit more accessible for creative applications. Kinect Common Bridge addresses an audience of creative individuals who don’t consider themselves “developers.” It emphasizes simplicity over features and enables those without advanced programming knowledge to become productive quickly.

“Artists approach coding from a different mentality and need a different language,” George explains. “Microsoft was open to embracing the lessons we had learned from open-source initiatives and bringing that thinking into the platform itself. Simple programming interfaces and the generosity of an open-source community are what made the Kinect platform popular with creative coders when it was released, and Microsoft understands how crucial that is for the adoption of Kinect as a platform for digital art.”

For George, this project symbolizes a crossover between disciplines. He points to work by Joshua Noble, who collaborated with George to develop an add-on that integrates Kinect Common Bridge with the popular openFrameworks creative coding environment as an example of how devices are being used for purposes other than their original intent.

Where Art Meets Science

George always has seen the integration of art and science as a way to push creative boundaries.

“What do we associate with the terms ‘engineer,’ ‘programmer,’ ‘researcher,’ or ‘artist’?” he asks. “The terms relate to intention, but the tools we use can be identical. The property of having a shared craft but different intentions provides grounds for collaboration between all these disciplines. It’s the idea of interrogating culture and looking at how things affect us. For me, that’s at an emotional-response level. I use technology not for an immediate utilitarian purpose but toward more poetic results, which can be as profound as new inventions but which resonate at a different level. They speak to you in the way that hearing a beautiful song or seeing a painting can resonate.”

At the same time, he has discovered even more similarities between the processes of creating art and performing research.

“Before this residency,” George says, “I had not been aware of the extent and makeup of the peer-review process in research, but there’s another analogy to art: Both are based on interacting with a community that shares a similar goal or similar ideas, and each person makes a unique contribution.”

Researchers and the AiR Experience

What do researchers regard George’s time at the lab? Loop, who has worked closely with the artist, can’t say enough about the experience.

“James is fluent in computer technologies and a master at cobbling together publicly available tools and code fragments, usually at a fairly high, abstract level, to create something interesting and beautiful,” Loop says. “He’s helped me think about my current work at a higher level and how I can make it more accessible to others. A second benefit is that his connection to the digital-art world may expose these ideas to an influential community hungry for new ideas.”

Wilson also says the exchange of ideas has been inspiring.

“James brings a fresh perspective to our work,” Wilson explains. “I feel I learn new ways to think about our work every time we chat. He backs his eloquence as a thoughtful artist with an ability to realize his visions in code, which means a lot in a research organization that is focused on realizing the future by actually building it.”

What lies ahead for Studio 99 and the AiR program?

“As we think about how an artist in residence can engage and inspire us, James has shown us how it can work really well,” Wilson says. “Our experience with him has us eager to continue the artist-in-residence program.”