(This story was corrected to show that the developer kit will be available at a later date.)
Matt Bell moves about an elevated podium in a conference room overlooking San Francisco's Market Street, using a camera on the Microsoft (MSFT) Kinect to snap photos of the podium from different angles. Bell, 31, has hacked into the machine—originally designed for playing games on Microsoft's Xbox gaming console—and is using it in sync with a personal computer. He's showing his creation to a crowd of 50 rapt software developers.
He aims to harness Kinect's 3D camera and sensors to craft software that can be used by real estate agents or people selling items on EBay (EBAY). Bell is among hundreds of programmers using the technology to build everything from games that help people cope with Parkinson's disease to tools that let robots detect obstacles.
"Hackers love to experiment with new technologies and see what is possible with them—and there is just so much that has become possible, thanks to the Kinect," Bell says. "The Kinect is the first consumer-level device that can see in 3D."
Some hackers can wreak havoc with gaming consoles and the systems connecting them, as Sony (SNE) learned after cyberattacks on the PlayStation network led to the theft of data on more than 100 million accounts. Hackers targeted Sony in April in retaliation for the company's effort to keep programmers from tinkering with its PlayStation 3. The incident may cost about 14 billion yen ($173 million). Nintendo, maker of the Wii gaming console, says it, too, was targeted in an online-data attack, although it lost no personal or company information.
Microsoft, by contrast, stands to benefit as developers such as Bell pave the way to wider use of its motion-sensing technology.
Record Sales for Kinect
The Kinect, which lets users play digital games with their bodies, is history's fastest-selling consumer electronic device, according to Guinness World Records. That's helping to boost demand for the Xbox: Sales in the unit that includes gaming consoles rose 14 percent to $1.9 billion in the March quarter, making up 12 percent of total company revenue and helping Microsoft compensate for weakness in its online division, as well as sluggish adoption of the new version of Windows for phones.
While developers such as Bell tinker with Kinect without obtaining Microsoft's blessing, the company has said it will soon release the code to let programmers build on the technology for noncommercial uses. A kit for developers who want to build products for sale will be available at a later date. "Kinect represents the first incarnation of the next big thing in computing—a world where computing is becoming more natural and intuitive," Stephanie Wettstein, a spokeswoman for Redmond (Wash.)-based Microsoft, said in a statement. She declined to specify release dates.
Bell, who holds a computer science degree from Stanford University and formerly worked for Google (GOOG), didn't want to wait. He assembled about 10 hackers for a December meeting in Menlo Park, Calif. Their ranks had grown fivefold by the time of the May 24 meeting above the Westfield Mall, near San Francisco's Union Square. There, they tap away at open laptops perched on rows of tables while Bell manipulates images of the podium into a 3D image that can be viewed from any angle. "We can now capture 3D in an affordable way," he says. "Previously, if you wanted to reconstruct a scene in 3D, you would have spent thousands of dollars."
Who Needs a Keyboard and Mouse?
The device's $150 price tag makes it cheap for 3dcapture.it, the start-up Bell co-founded, to overcome the challenge, he says. Software developers can use 3D technology to alter the way people interact with a personal computer, says Andrew Tschesnok, chief executive officer and founder of Organic Motion, a software company that has experimented with Kinect cameras. "If a computer can see you, then you don't have to use a keyboard and a mouse to interact," he says.
Until now, PCs have largely functioned like one-way windows. Motion-sensing technology helps Microsoft update its software as people migrate away from traditional machines, says Sarah Rotman Epps, an analyst at Cambridge (Mass.)-based Forrester Research (FORR). Clothing retailer Nordstrom (JWN) used a hacked Kinect for a window-light display at its flagship store in Seattle, letting customers who approached the window wave their hands and write words in light on the window.
"This was the first time we sought to engage customers with this type of technology and it's in keeping with how we want to experiment and try different things," says Colin Johnson, a spokesman for Nordstrom.
The Kinect system has two cameras. One combines with an infrared projector to acquire 3D information, which is then sent to an image processor supplied by Tel Aviv-based PrimeSense. Researchers at the Hybrid Systems Lab at the University of California, Berkeley, used the Kinect as an inexpensive way to give an aerial drone robotic vision to navigate. Evoluce, based in Hallbergmoos, Germany, has eliminated the need for a computer mouse and lets workers control Web browsers, build PowerPoint presentations, and manipulate other Microsoft business-productivity tools with hand gestures. Similarly, surgeons at Sunnybrook Hospital in Toronto can use operating room computers without touching them, thanks to the efforts of hospital engineer Jamie Tremaine.
Avoiding Forklift Accidents
One of the presenters at the Kinect hackers' meeting was Erin Rapacki, who works for Adept Technology (ADEP), a maker of robots for factories. Rapacki says that Kinect cameras could help forklift operators avoid accidents. The U.S. Occupational Safety & Health Administration estimates that there are about 110,000 such accidents annually. Currently, forklifts use laser-based sensors that can cost $5,000 apiece and can scan only a single plane, Rapacki says. Kinect sensors work in three dimensions.
The race to find other uses for Kinect dates back at least to Nov. 4, when Adafruit Industries, a New York company that sells tools for open-source electronics projects, offered $3,000 to the developer who could create the first open-source software that would let developers create applications on top of Kinect. Microsoft said at the time that it "does not condone the modification of its products."
The company has since changed its tune. Microsoft has posted ads for developers whose job will be to integrate Kinect into Windows. It has created a new group called Kinect for Windows within its Interactive Entertainment Business division, which is responsible for designing Kinect-related tools. "Microsoft has been supportive," says Bell, as several hackers nod in agreement. When the hacking group outgrew its Silicon Valley meeting location, Microsoft offered its San Francisco offices for the May 24 meeting. The podium that Bell photographs on that occasion happens to feature a Microsoft logo.