Computing That Only Looks Like Child's Play

Can MIT replace keyboards with pinwheels, globes, and what resembles a hockey game?

It would be easy to mistake Hiroshi Ishii's lab at Massachusetts Institute of Technology for a day-care center after all the toddlers have scampered through. Listless pinwheels hang above a cubicle. A sandbox sports a huge crater, as if Big Foot had visited. Projectors dangle from the ceiling at precarious angles, and small wooden houses lie strewn across tables.

These seeming remnants of child's play are, in fact, the tools of a very adult quest to forge a new frontier in computing. For the past eight years, the 45-year-old, Japanese-born Ishii has been striving to liberate society from the tyranny of computer keyboards, monitors, and mice by creating new ways for humans to view and manipulate data. A silicon-enhanced pinwheel turning furiously, for instance, might warn a data-center manager that a network is nearing overload. Or homebuilders might mold sensor-laden sand into different shapes and patterns on a table to evaluate drainage in an area targeted for development. Microprocessors would still be running software models to crunch the data -- but users would never have to touch a keyboard. "We're inventing a new medium," says Ishii. It looks nothing like the windows, icons, and pull-down menus that make up today's graphical user interface. "But in certain areas, like teaching or design," Ishii declares, "our interface is superior."


A decade ago, a handful of evangelists like Ishii talked about "tangible" computer interfaces that could engage the user in a more physical way. In 1995, Ishii's Tangible Media Group was created to do just that. And while the science on display here sometimes appears fanciful, this isn't a playpen where engineers can indulge their every digital whim. The research is grounded in the real world, and it has attracted investment from companies such as Intel, furniture maker Steelcase, and Japanese telecom colossus Nippon Telegraph & Telephone. The lab brings together teams from industry and academia, and the ideas have plenty of support outside the Media Lab. "We ought to use the power of computing to get the technology the hell out of the way," says John Seely Brown, the former director of Palo Alto Research Center, which did similar work in the 1990s.

Ishii's journey toward tangible computing hinged on one critical encounter. As a researcher at NTT (NTT ) in the late 1980s and early '90s, he worked on souped-up videoconferencing systems. In 1994, he presented his work at a conference, and MIT Media Lab co-founder Nicholas Negroponte heard him speak. Right away, says Ishii, "he challenged me to stop my previous work. He doesn't like incremental things. He said 'life is short."' It took Ishii about 10 minutes to make up his mind. Just a year later, he moved to Cambridge, Mass., and quickly started making his mark in the area of next-generation interfaces.

Now, Ishii is getting a chance to test-drive one of his most talked-about gizmos. Called IP Network Design Workbench, it aims to make it easier for engineers and nontechnical execs to collaborate in designing telecom and computer networks. It does this by letting team members sit around a "sensetable" and rejigger components of the network -- routers, servers, storage systems, fiber optic pipes, and the like -- and examine the results of their experiments in real time.

The project's sponsor, an NTT unit called NTT Comware, is counting on the workbench to help recruit new network customers outside NTT's family of companies. Running a data network with 60 million subscribers is a $3 billion-a-year business for Comware. But to grow, it wants to design and sell Web-based networks -- from virtual private networks to entire telecom systems -- to other companies and government agencies. "Within NTT, it's easy to communicate what a network is," says Kazuhiko Tanaka, a Comware general manager based in Cambridge, Mass. "Outside, we have to explain."

How will the table help? The power is all in the user interface. A big-ticket purchase like a network -- or even a network upgrade -- often requires an O.K. from a company's chief financial officer, CEO, or both. These execs typically aren't comfortable using computer-aided design programs running on workstations to grasp the merits of what they're buying. Tanaka thinks that if the customer can sit at a table with Comware engineers and see how changing the number or location of network routers and servers will effect performance and costs, they'll be able to settle quickly on a design and close the sale.


Right now, the demo table at MIT displays a relatively simple configuration of switches and computers typical of a small office. A video projector above shines a network design onto a 2-foot-by-3-foot white table. An antenna embedded in the table can track the movement and placement of electronically charged objects, resembling small hockey pucks, that slide over the table surface. Using the pucks, Tanaka can shift icons projected onto the table to change the network configuration. He might, say, add an e-mail server or upgrade switches and routers. In a flash, the computer that drives this simulation reconfigures the network, and the picture and additional displays adjust to tell the user two things: how far the upgrade will go to meet expanded requirements and whether it can all happen within budget.

The software involved in this simulation isn't so different from the programs that run on a workstation. Based on input from the table, a program crunches the numbers and relays the new network specs to the projector overhead, which is wired to the same microchips that handle sensory input from the table. Network designers at the table can immediately see the impact of their work. A line representing bandwidth might change color from blue to red, signaling a need for more capacity. That means higher cost, with an estimate appearing on a screen attached to the table. "You could prepare beautiful PowerPoint slides, but it's dead ink," says Ishii. "With the table, you're engaged. It's an easier sell because the customer is a part of the design team." NTT Comware expects to start using the system next year in Japan and hopes to sell it to other major telecom carriers as well. Next, Tanaka and his team hope to create a truly portable version that can be carried around on sales calls.

Ishii's team is working on similar systems for urban or landscape design. MIT design students, acting as product testers, have begun building miniature 3D towns on the table, enabling them to gauge changing traffic patterns, airflow, and the movement of shadows during the course of a day. The developers can also use another gizmo that looks like a sandbox to help make decisions about where and how to build. The user manually shifts the "sand" -- semi-translucent glass beads -- to alter slope or contours. An infrared light shines from the bottom of the box, allowing a camera overhead to distinguish shapes on the table. The camera feeds the 3D geometry into a computer, which analyzes the data, then shoots back color-coded markers that tell the direction or speed of water flow, for instance.

Ishii's lab isn't the only one trying to make data tangible. Elizabeth Mynatt, an associate professor of computer science at Georgia Institute of Technology, wants to make data a "warmer" presence in the home. With support from Motorola, Hewlett-Packard, and Intel, among others, she developed a digital family portrait that electronically links members of a real family. Say you'd like to keep tabs on your aging mother, who lives alone. A sensor network in Mom's home could discreetly monitor her activity and send data to a reader in your home -- which might be a digitized frame around a photo of Mom. Pictures of butterflies that stud the frame grow larger or smaller based on her activity level. If the butterflies diminish drastically in size, you'll notice and give her a call. Without being too intrusive, "it gives an overall sense of how she's doing," says Mynatt.


Applications like this may still be a few years off. But shoppers at Brookstone Co. (BKST ) stores can already see what tangible computing is all about. On display is a product called the Ambient Orb -- a $150 glass globe that plugs into the wall and receives wireless data feeds of stock activity, traffic, weather, or other types of information you select. The orb changes color as the data change. So green could mean that the I-93 artery heading north from Boston is clear; red means it's clogged.

Ambient Devices, the startup in Cambridge, Mass., that developed the orbs based on work done in Ishii's lab, has more elaborate ambitions. It's in talks with British telecom company Orange Innovations to supply similar devices to customers. Ambient says it's also negotiating a deal with Bloomberg LP, which might use ambient pinwheels to monitor the performance of regional sales teams. "The concept could become much more widespread," says Negroponte, who has invested $150,000 in Ambient and sits on its board. "But there is still a great deal of work to be done."

In the old days, the kind of product development work that Negroponte is talking about would undoubtedly involve lots of engineers with eyes locked on computer screens and fingers flying over keyboards. But as Ishii's ideas emanate beyond his laboratory and tangible bits replace the one-dimensional variety, there's no telling what form human-machine interactions are likely to take next.

By Faith Arner in Cambridge, Mass.

    Before it's here, it's on the Bloomberg Terminal.