A Keyboard and Mouse? That's So 1970s

Editor's Note: The annual TED Conference (Feb. 26Mar. 4) is an invitation-only affair where high-tech tycoons, Nobel laureates, and other very smart people gather to share ideas that will inspire. The TED Fellows program was established to give people who wouldn't ordinarily have the opportunity—or the means—to present their remarkable work to an audience that just might include Bill Gates and Al Gore. In a series leading up to TED, will feature interviews conducted via e-mail with a handful of this year's fellows.

James McMichael Patten
Interaction designer, inventor
Founder and CEO, Patten Studio

James Patten has a modest goal: He simply wants to change the way people use computers by doing away with the ubiquitous keyboard and mouse.

Patten, who has a PhD from the MIT Media Lab, has proved that we can get along without those artifacts, having created several projects for corporate clients and museums that employ common objects to represent and control data inside a computer. One of his projects, a Geiger counter-like device that gives shoppers the environmental record of that company, was shown at the Museum of Modern Art in New York.

Last year, the 33-year-old's Brooklyn-based design firm produced "Create a Chemical Reaction," an interactive exhibit about chemistry for Chicago's Museum of Science and Industry. Using two eight-foot long interactive tables, visitors could grab atoms (represented by physical objects) off of the periodic table and bring them together to cause chemical reactions. One day, that may be just how students study chemistry.

Q: What is the most unusual object you've made interactive?

A: I made an ethical shopping aid in the form of a Geiger counter called the Corporate Fallout Detector. The device has a database of ethical and environmental data about companies that make consumer products. When you scan a product with its barcode reader, it emits a clicking sound (like a Geiger counter) based on the ethical or environmental record of that company. This project underscores the power of physical objects to convey emotion in a more powerful way than a software application on a screen can. It really attracts attention when you go shopping.

You use it when you go shopping?

It attracts some stares from other shoppers, both curious and apprehensive. Some people have come up to me and asked what it is, and when I explain it to them, they usually get excited and ask if I'm planning to market it. I do think there is a market for something like this. If it caught on, it could potentially change the way some companies look at corporate social responsibility. There are a few ethical shopping apps for smartphones, but in my opinion there is still room for a great app in this category.

Has the ubiquity of smartphones, devices like the Wii, and now tablet computers, changed the way people interact with computers?

The Wii shows that when we rethink interactive technology in terms of what is easy, intuitive, and fun for people, it is possible to create a tremendously compelling experience. The power of this experience helped Wii get all kinds of new demographics excited about computer gaming. Touchscreen-based devices like the iPad are also a step in the right direction. The success of the Wii and the iPad show that there are tremendous opportunities in the market for products that abandon traditional modes of interaction.

What do you have against the keyboard and the mouse?

Many of the things people want to use a computer for today aren't a great fit for the keyboard and mouse. The keyboard and its ancestor, the typewriter, were designed for writing text, not for browsing photos, editing video, sketching, etc. There are so many ways that we use computers now that could not have been imagined in the era of the typewriter. These new applications deserve a rethink of how we interact with computers.

One approach I've explored in depth is using physical objects on a tabletop to represent and control information inside the computer. This approach lets people organize information spatially and leverage a rich set of skills that humans have for using our sense of touch. In one application we made, physical objects on a table represent various parts of a manufacturing supply chain. By moving and rotating these objects, one can control a simulation of that supply chain, for example, by changing the output of a factory. You can do the same thing with a keyboard and mouse, but doing it on an interactive table is easier to understand and creates a more effective learning experience.

What are you currently working on?

One project I'm particularly excited about is an interactive tabletop system for visualizing and interpreting large volumes of financial data. We use physical objects on an interactive surface to represent concepts in the system, such as a particular class of assets or window of time. By moving these objects on the surface, our users can ask more sophisticated questions and get faster answers than is possible with more traditional approaches to accessing financial data. We think this approach to interaction has a wide variety of applications, ranging from education to urban planning.

Is there any one object you'd like to digitize that you haven't had the chance?

Compared with the computers we use today, musical instruments have "user interfaces" that are incredibly refined and expressive. A guitar is simple enough that a five-year-old can look at it and understand how to make sound, but sophisticated enough that it rewards many years of practice with more expressive power. Not everyone has the time to master the guitar, but more and more of us are spending thousands and thousands of hours with our computers. I believe that if we apply the same design principles that are used for musical instruments to the design of computers, we can make them more fluid, expressive, and intuitive as well. When I think about what it should feel like to interact with a computer, I often turn to musical instruments for inspiration.

Why don't we see interactivity built into more everyday objects, like the refrigerator, for example. In many households that's where you'll find all the important "data" simply taped to it.

The everyday refrigerator provides a much richer interactive experience than any computer-embedded refrigerator prototype I've seen. The refrigerator door is a blank canvas for a family to post photographs, notes, cards, etc. in a fun way that everyone understands. It does its job so well that it's hard to see how an embedded computer can really improve on it. I think sometimes making an experience digital can take away more than it adds.

    Before it's here, it's on the Bloomberg Terminal. LEARN MORE