Boston-based company Sosolimited extracts the hidden meanings from presidential debates and visualizes them. Co-founder Eric Gunther explains his strategy.

Eric Gunther loves Swiss design. On his desk at Sosolimited, the company he established in downtown Boston two and a half years ago, lies a copy of Josef Müller-Brockmann’s book Grid Systems in Graphic Design. “That’s our bible,” he smiles. “It’s so good.”

Eric is a computer scientist – turned designer and artist. He and his two partners from Sosolimited, Justin Manor and John Rothenberg, are MIT graduates who all worked at the same Boston design firm, specializing in interactive displays for museums and corporate clients. In 2003, they formed an art group that introduced a novel kind of show: “remixes” of political debates, where speech data is analyzed and visualized live. Five years later, the Institute of Contemporary Art in Boston invited them to present the Obama-McCain debate. Their production – which was widely remarked upon – looked like a Kraftwerk performance: the three artists were sitting in front of the audience wearing suits and sunglasses, controlling live the audiovisual projection just behind their backs.

Today, Sosolimited is a five-person company working at the intersection of data visualization, art and technology. It produces interactive installations for clients such as IBM, Chevrolet and LG Electronics.

You started Sosolimited by remixing live political debates. What do you plan for this year’s presidential debates?

This year we’re changing our approach. We’re going to create a distributed online experience that will be on the web as well as on mobile platforms. The idea is to let people drive a console that streams the speech’s text live and visualizes it with respect to criteria chosen by the user, such as emotional content or specific themes.

People will be then able to publish their own view of the data onto Facebook or Twitter. Press coverage of the debates is intensely biased and part of our idea to counteract this is to provide people with a totally data-based, unbiased view of the events, and allow them to share their own version. At the same time, the piece will retain an experimental and playful feel. In other words, the material we are working with is data and information design, but we view the final product as a piece of art.

How does it work? Do you have to convert audio stream into text?

We don’t have to convert it – we use the closed-caption stream. That’s the text embedded in the TV signal for hearing-impaired people. We feed that into our analysis software and then extract interesting patterns, like the frequency of word usage or the emotional state of the speakers. It’s amazing how you can figure out a lot of things just by looking at simple words such as articles like “a” or “the”. For instance you can determine how vague, how categorical or how feminine someone is being, or how “presidential” the candidates sound. This linguistic analysis is based on the research of a group at University of Texas, Austin, led by James Pennebaker.

You could apply similar technologies to the analysis of meetings in companies.

People are using this research in the finance world. Language analysis engines have been used to go through annual reports and materials published by companies in order to extract hidden meanings. They then use the results to make financial decisions. There are also companies analyzing customers’ emotional reactions to products on Twitter. Social media analytics is big business right now. You can pull an increasing amount of meaning out of streams of opinions and information on the web.

So you started Sosolimited with the debates business…

That was before we even started the company – we were doing it at night while working at Small Design Firm! We would stay late and continue coding into the evening. At first we were an audiovisual VJ/DJ performance group.

Do you all code?

We’re all programmers and designers. We have various backgrounds: I studied computer science, John trained as an architect, Lauren studied computer science and visual arts and Justin started with physics and then studied with John Maeda at the Media Lab. We cut our teeth programming while working at Small Design Firm. There we were also mentored by a gifted designer named Timon Botez. Chen, our designer at Soso, is the only one of us with formal graphic design training.

What are you working on now?

We’re working on a piece for the Olympics right now. It’s top secret, but we’re pretty excited about it. We have just finished a project at the North Carolina Museum of Natural Sciences with Jeff Lieberman and Bill Washabaugh. It’s a very big sculpture that hangs from the ceiling in the atrium of the museum and is composed of 3’600 LCD glass pieces with independent transparency control. It continually displays natural patterns like fish, birds, bacteria and different kinds of water surfaces. There are all sorts of reflections and shadows giving it a very strange effect. And it’s super low power: the whole thing runs on less power than a laptop. Jeff and Bill managed to use the structural cables to supply power and data to the sculpture so it just looks as if it’s floating in space. It’s quite beautiful.

Is it art or scientific visualization?

It sort of sits at the boundary between art and science. It’s in a science museum and it’s all based on scientific content. There is a screen that tells the visitors what’s currently happening on the ribbon. The goal, though, is to inspire awe in visitors to the museum and to tap into the same emotional well that fuels their interest in science. So in this way, it is a piece of art.

How long were you working on that?

That was a long project, lasting three years, on and off. We started with David Small and continued working on it with Jeff and Bill at Sosolimited.

Does it work at night?

At night it is lit with spotlights, which makes for some interesting shadows. In the dark, you can see it from all the way out on the sidewalk, about a hundred yards from the museum. It really shines in natural sunlight, though.

Do you do data visualization?

We haven’t done so much commercial data visualization work. That doesn’t stop us from casting a critical eye over much of the data visualization we see, though. A lot of the data visualization we’ve done has been for our own artistic projects such as the debates and the TV remix work.

Do you see a business emerging for the kind if work you do?

Somehow we’ve been able to maintain the business through quite a variety of projects! We started out doing mostly museum installations but now we’ve produced social network interfaces online, sculptural pieces, interactive installations… We’re all over the place. I guess that our education taught us to think about a problem from an engineering point of view and figure out how to solve it.

What would your dream project be? What direction do you want to go in?

I think our dream is to do 100% commissioned, self-directed artwork, where we are both the creative directors and we create the work as well. I think that’s the dream. As far as the kind of things we’re into, I don’t think there is any one specific direction. The television work has been fruitful, but we’re also interested in pursuing more sculptural work, moving away from the small screen.

Is there anybody else doing similar work here in Boston?

There is our former employer, Small Design Firm. Some friends, also MIT guys, have a company called Zigelbaum & Coelho – they’re a little bit more involved on the hardware side of things. There’s also our friend Jeff Lieberman, who’s less involved with graphics and data visualization and more with kinetic sculptures. And of course, Ben Fry runs Fathom Design a few miles away from our studio.

And elsewhere in the world?

One of our heroes in this field is Art+COM in Germany. They’ve been around for a long time and do inspiring work that has continued to push the boundaries for two decades.

What do you call your field of activity?

Some people call it interactive design but that doesn’t really work anymore because people use this term for the web. Sometimes we use the funny word “Infotainment”. It’s a terrible word but it does express the idea of taking a more playful approach to information design. With the explosion of information design comes a mess of bad visualizations that misrepresent data. We’re interested in playing in that field: for example, when is it ok to play around with the data? How serious do you have to be?

So you don’t always try to stay strictly faithful to the data?

For our art pieces at least, no; and I think that’s confusing to people sometimes. Often, at the end of the political debates, people ask, “Who won?”. They see numbers and data visualizations and immediately their minds jump to some sort of data mode. They want facts; they want a rank, they want to know who’s the best, and who’s worst.

And you’re not into that. You’re more into letting them explore.

It may sound wishy-washy but I think we’re more interested in what happens to the consumption of media when you introduce the data as a stream with the media. Usually data visualizations refer to things that are miles away. Take an infographic about oil: you can’t see the barrels the data is referring to. In television, information comes at you in real time and we show the data right alongside it. We’re interested in how that affects your consumption of the actual media, the full resolution media.

What other projects have you done?

A firm called OpenEnded Group motion-captured Merce Cunningham while he was still alive and created this beautiful visualization with his fingers while dancing. A few years later they open-sourced the data to a bunch of artists and Boston Cyberarts commissioned five artists to reinterpret the data. So we did this typographical reinterpretation where we took articles like reviews and critiques about Cunningham, and we mapped them to the paths of the fingers. The movements of his fingers are actually spelling out the articles in the 3D space of the screen. And his hands are driving the camera. So depending on how he’s moving his hands, the camera moves around the space.

I guess the art we do uses more the language of design and information design as our medium, mainly because that’s what we were all working with and exposed to. That became the material for us as artists and that’s why a lot of our work looks like it could be a design piece.

It’s funny that you all have technical backgrounds (MIT) but that you refer to yourselves as artists.

Well it depends on the time of the day. Often the engineer persona takes over, and even during the artistic process the engineer in us is always present, trying to solve problems and define things. Sometimes it’s an interesting balancing act between the two roles.

Sosolimited’s website