Source: Microsoft

Here's What It's Like to Build Hologram Apps for Microsoft's HoloLens

You will love my holographic birds

A big part of Microsoft's pitch for HoloLens, a headset with glasses designed to project interactive 3-D holograms, is that developers will easily be able to make any Windows program compatible. That would ensure plenty of software available for HoloLens at launch.

On Thursday, Microsoft gave me a chance to experience the latest version of HoloLens (wireless, finally), as well as to try out my limited programming skills on the company's developer tools. While my expertise in building apps extends as far as one programming class in college (it fulfilled the same requirement as calculus), I was able to, with the help of some very patient Microsoft technicians, create a serviceable app that responded to my voice and touch.

The demonstration was pegged to Microsoft Build, a developers conference in San Francisco that kicked off April 28. Microsoft decided to undertake the unenviable task of trying to teach a roomful of reporters how to build a holographic app. Call it Coding for Dummies, er, tech journalists. Each pair of us were assigned a "mentor" to keep us from messing up too badly. To condense what is normally a four-hour course into 90 minutes, Microsoft wrote the scripts for us. They also designed the art components ahead of time, which is helpful because I'm about as terrible at art as I am at programming.

Source: Microsoft

First, I was instructed to set up a hologram of a rectangular space containing various origami paper objects. After plopping in two paper airplanes and two spheres, I added a cursor and set it to move based on which way my head faces while wearing the headset. Next, I set the balls to drop when the user taps a finger in the air in front of the glasses, which is a common gesture that controls many of the actions in apps for the upcoming Windows Holographic software. In my app, a voice command will also tell the balls to drop. Just to be a bit fancy, I went into the code and changed it to respond to my own choice of words. Finally, I added an ambient music track to play over the HoloLens's built-in speakers, which increases in volume as I get closer to the hologram, and programmed the spheres to sound like crumpling paper as they rolled.

The first version of my app was flawed. When I did my finger tap, the balls would fall and roll out of the holo-world I'd created, gone into the holo-abyss. So I was instructed to go into the spatial mapping settings and turn on the ability for holograms to interact with the real world around them. Suddenly, the whole room I was standing in—the floor, tables, chairs and other people—was part of the field of play. Then I learned how to move my holograms. For example, I could affix one to my mentor's head. The holograms are incredibly responsive to their surroundings. When I placed mine in the air, one ball dropped and got stuck in a nook on the real-life desk chair underneath. My mentor moved the chair, and voila, the ball dropped to the floor. 

There's an option to create an underworld, essentially a hologram land beneath your feet. In January, Microsoft demonstrated HoloLens with a version of the computer game Minecraft, which the company acquired last year. In it, I was able to blast through a coffee table and see the Minecraft layer below. I added a similar feature to my own app, where users could use the spheres to blow a hole in the floor, revealing a world of green field, river, clouds and red origami birds flying around. Those in the room without a headset on got to see a bunch of journalists crawling on their hands and knees.

Microsoft's Joe Belfiore, left, tries on a HoloLens device with colleagues Alex Kipman, right, and Terry Myerson following an event demonstrating new features of Windows 10 at the company's headquarters on Jan. 21, 2015, in Redmond, Washington.
Microsoft's Joe Belfiore, left, tries on a HoloLens device with colleagues Alex Kipman, right, and Terry Myerson following an event demonstrating new features of Windows 10 at the company's headquarters on Jan. 21, 2015, in Redmond, Washington.
Photographer: Elaine Thompson/AP Photo

The good: It's really cool, and I can already see the potential for what kinds of inventive worlds real developers can create. Unlike the headsets and glasses from the January demo, the new wireless version means Microsoft employees no longer have to follow me around, trying to keep me from tripping over the wires while carrying on a holographic Skype conversation. The headsets Microsoft showed on Thursday are closer to the final version, which we still don't have a release date for.

The bad: The field of vision on the glasses is still somewhat narrow. Move your head too much, and your hologram disappears. There is no holographic peripheral vision. The headsets are also pretty sensitive to the movement of your head. Since your head controls the cursor, moving slowly and fluidly took some getting used to.

But the product has shown significant progress just over the last three months. Walt Disney and Unity Technologies, a game engine developer, are among the big names creating software for it. "HoloLens is real. This isn’t fiction. This is science fact, not science fiction," says Terry Myerson, executive vice president of Microsoft's operating systems group. "We have to get the price down; we have to get the form factor right for it to be a large-scale breakthrough."

Before it's here, it's on the Bloomberg Terminal. LEARN MORE