Making Some Wii Modifications

Johnny Chung Lee has achieved some fanfare modifying Wii's attributes. Now he'd like to bring touch- and motion-based interactivity to your desktop

Over the past 12 months, a series of quirky but compelling videos uploaded to Google's (GOOG) YouTube have been delighting hackers, designers, and tech tinkerers worldwide. The videos, which feature modifications of Nintendo's (NTDOY) popular Wii console to create everything from mind-boggling 3D images to interactive whiteboards, have earned their creator a cultlike following and inspired countless other experiments.

The four- to five-minute films are the handiwork of Johnny Chung Lee, a 28-year-old graduate student at Carnegie Mellon University. Lee, who was recruited to the university by the well-known lecturer and computer scientist Randy Pausch (, 11/21/07), will earn his PhD in human-computer interaction later this spring.

He gravitates toward projects that recreate the intricate interactions found in hot-selling devices including Apple's (AAPL) iPhone—but at dramatically reduced cost. His experiments demonstrate how simple, inexpensive modifications of common gaming technologies could be used to foster collaborative innovation in a business or research environment.

BusinessWeek reporter Matt Vella recently spoke with Lee about the common themes in his research, the future of motion and multitouch technology, and why exactly he chose to use the Wii as a vehicle for some of his experiments. (For examples of Lee's work, see the video slide show.)

What attracted you to the Wii originally?

A few years ago, I was playing with accelerometers [gyroscopic devices that sense motion]. These are very interesting interaction devices that researchers have been looking at for a couple of decades and that could always do neat things. Seeing one in a game device was a little bit overdue. A few years ago, when I was doing research with Microsoft (MSFT), I was one of several people trying to get the Xbox group to include an accelerometer in the controller, but they decided not to for cost reasons.

So when the Wii remote came out and it became public that there was a motion sensor in it, I was very excited about it. And then the development community started reverse engineering it and discovered it had all these other capabilities, like an infrared camera, an expansion port, and pressure-sensitive buttons—and it hooked up to a computer relatively easily. All of those factors, plus the low cost, the high availability, the high capability, and its ease of use, made it very attractive to play with.

So it's really the technology in game machines that interests you more than gaming per se?

My interest is interaction techniques and interaction capability. So I'm more interested in systems or tools that allow you to manipulate data or a computational system more effectively or efficiently. Gaming technology just happens to be one of the more rapidly evolving platforms for interaction techniques. On the desktop, you're kind of stuck with the keyboard and mouse, and it's very hard to convince people to move away from that. The cultural environment in gaming is much more dynamic, so almost anything goes. People are much more willing to adopt new ways of controlling a game, as we've seen recently with the Wii as well as games like Rock Band and Guitar Hero.

Do you feel that your projects share a common theme?

I try to look for simple opportunities that can dramatically change the distribution of technology. One way of doing that is by making things much simpler or much cheaper. For example, the Wii remote can dramatically reduce the cost of the whiteboard.

Similarly, my other projects tend to have this motivation—simple ideas that end up having a relatively big impact because they drastically change the economics of doing a particular task.

Why is this particularly appealing to you?

If, suddenly, a certain technology or certain capability is affecting 10 to 100 times as many people as it was before, it's much more powerful. I find that to be a valuable, visceral, and very measurable impact of my work. When I do something and so many other people begin experimenting because of what I've done, well, I find that very rewarding. YouTube has actually been one of the better ways of measuring that because I put my videos up and within days or weeks people post their own videos of things they've created based on my work.

Is there something about being a grad student that may have made you predisposed to try to do more, more cheaply?

I think that does play some role, but I don't know if it's a major one. I only pick projects that I can execute. Certainly sometimes I don't have immense resources to accomplish certain projects. That's not to say if I wanted to they're not available. For example, at Carnegie Mellon there are $500,000 motion capture systems, and there are MRI machines at the University of Pittsburgh Medical Center that I could use. But those have not particularly been exciting to me because I know that even if I did something interesting, the number of people that can benefit from that is very small. The major factor is projects that could affect a lot more people.

Where do you think the greatest possibilities for touch and motion-based interactivity lie within traditional computing environments?

Finding interesting new interaction techniques that have a high-yield impact on productivity applications is the most desirable for me, even though the keyboard and mouse is pretty efficient. New interaction capabilities are going to apply mostly to esoteric applications. For example, these multitouch systems are not so great for helping you with Excel—but they definitely help for things like navigating maps and are great for manipulating photos or navigating large datasets. Multitouch is probably overdue in a lot of Adobe (ADBE) products because it's very good at creating images.

So what's next?

I would like to continue doing research because I enjoy the freedom. Computers in general are probably one of the most versatile tools for doing just about anything. So if you can improve the way you interact with a computer, you can have a big impact. To me the computer is a stepping stone to solving most of the world's problems. As someone recently said to me—the future is going to be awesome—I think there's some truth to that.

Before it's here, it's on the Bloomberg Terminal.