Look Ma, No Hands

I can't help noticing that "Ralph" steers like my brother-in-law Tony. As we pick our way through early morning traffic on the interstate highways around Kansas City, Ralph often jerks the wheel. It's a bit annoying. But I guess you have to expect some quirks in a digital driver that's barely five months old and steers our 1990 Pontiac Trans Sport minivan with the aid of just one eye: a video camera pointed at the highway from behind the rearview mirror.

I'm tagging along on an epic trip. By day's end, the computer program known as Ralph will autonomously navigate for 98.7% of the 644 miles across the Great Plains from Blue Springs, Mo., to Denver--the farthest ever traveled by a "smart" vehicle in one shot. Today is the longest leg of a 2,797-mile journey from Washington to San Diego, dubbed "No Hands Across America" by the Carnegie Mellon University researchers who put Ralph together.

DRAMATIC DEMO. The No Hands trip is more than just a gimmick. It's both a test and a dramatic demonstration of the state of the art in automated steering. (The computer can't yet handle the throttle or the brakes.) Last October, the Transportation Dept. awarded $202 million to the National Automated Highway Systems Consortium for just such projects. General Motors Corp. heads the 40-member consortium, which includes Carnegie Mellon as well as Bechtel, Hughes Aircraft, Martin Marietta, and the University of California. The goals: to reduce highway carnage and smooth traffic flow by developing intelligent roads and cars that drive better than most people do. Other projects include collision-avoidance sensors and systems that adjust stoplights to avoid gridlock. The contract calls for all these systems to emerge as prototypes by 1997, with deployment starting in 2001. Europe and Japan have similar projects.

It will be years before drivers fully surrender their grip on the wheel to the likes of Ralph. Still, by early 1996 some trucks may be equipped with a less ambitious prototype. It won't steer--just sound an alarm and vibrate the wheel when the vehicle begins to leave the road. Cars could have similar alerts as options by 1999. Ralph's support team from Carnegie Mellon's Robotics Institute--research scientist Dean A. Pomerleau and graduate student Todd M. Jochem--have formed a startup, AssistWare Technology, in Wexford, Pa., to commercialize technology to prevent so-called road-departure accidents, which kill nearly 15,000 people a year.

Before we even reach Interstate 70, Ralph "crashes." Fortunately, we're standing still. Pomerleau had pulled off just short of the on-ramp to prepare the vehicle for self-steering. Its global positioning system had to be calibrated to steer with the help of satellites. It was also time to switch on the data logger, which keeps a blow-by-blow record of Ralph's performance. But when Pomerleau presses the appropriate keys on Ralph's computer--a Sparc LX portable workstation from RDI Computer Corp. in Carlsbad, Calif., that nestles between the front seats--the screen goes blank and the hard disk halts. It winks back to life after a hasty reboot.

The computer crashes occasionally because all the electronic gear draws its power from the van's cigarette lighter, which is barely adequate for the task. Still, being able to run everything off the van's standard power system is a vast improvement over Navigation Laboratory 1, which Pomerleau helped build in 1986 with funds from the Pentagon's Advanced Research Projects Agency (ARPA). To accomplish a fraction of what Ralph does, Navlab 1 required a supercomputer, three workstations (table), and an on-board generator. Navlab 2, an Army Humvee, also needs a generator. For Navlab 3, Pomerleau outfitted his Honda with Ralph and tested it while commuting this past spring. Navlab 4 is basically a twin of Navlab 2. Navlab 5 is the Ralph-equipped minivan.

RUSH-HOUR DAZE. Pomerleau eases onto I-70 and glances at the small liquid-crystal display mounted above the glove compartment. It shows the view ahead from the video camera, plus graphics overlays created by Ralph. Pomerleau relinquishes the wheel when the curving blue dashes, indicating where Ralph calculates the edges of the road to be, line up over the actual edges in the TV picture.

Almost immediately, we're surrounded by commuters rushing to work. After several cars cut in front of us, I ask about the jerky, Tony-like steering. Pomerleau explains that this happens when Ralph momentarily loses sight of the lane markers ahead of the car that's changing lanes. When the rush-hour traffic gets really heavy, so many lane markers are obscured that Pomerleau has to grab the wheel often and override the computer.

Ralph's next test comes when we reach new pavement with temporary lane markings that Ralph has trouble discerning in traffic. Then we hit a construction zone, with plastic cones directing cars onto a strip of temporary pavement. Ralph is confused, so Pomerleau punches an "update" button that tells the computer to forget what it had been expecting to see and reevaluate the view ahead from scratch. In an eyeblink, Ralph finds the edge of the dark asphalt and repaints the LCD monitor to show it has learned the new roadway.

The Navlab team prefers to travel in the left lane because there Ralph won't encounter any right-hand edge stripes leading to an off-ramp. When the system can't find regular lane markings, such as painted stripes or reflecting studs, it can key in on other features: the edge of the road, dark lines left by tires, even the trail of oil spots down the center of most lanes. When all such signs vanish, Ralph shakes the wheel and beeps, alerting the human driver to take over.

Once we leave Kansas City behind, Pomerleau accelerates to 70 mph. After several cars pass us on the right, he speeds up to 75 mph and engages cruise control. We all settle back to let Ralph rack up autonomous miles.

Ralph steers by an ingenious bit of software. The computer ignores everything in the video image except a rectangular section of the roadway. The area that gets analyzed is 7 meters wide, or about twice the width of a lane. It starts 20 meters in front of the vehicle and extends out to 70 meters, typically, although the computer shifts the rectangle as needed: Just like a person, it looks farther ahead at higher speeds.

On the LCD monitor, this rectangular area is shown in perspective as a trapezoid outlined in red. It is carved into a grid with 32 columns and 30 rows. The computer inspects the rows sequentially, hunting for parallel features such as lane stripes. It detects curves by measuring the rate at which parallel features shift left or right. Each analysis of the entire grid takes about a tenth of a second.

FAST LANE. Ralph's program is so elegantly simple that it took less than a month to develop after it was suggested by Charles E. Thorpe, a senior research scientist at CMU's Vision & Autonomous Systems Center. One reason it went so quickly, says Pomerleau, is that it includes software developed for Alvinn, the brains of Navlab 1 and 2. Some $10 million in ARPA support has helped CMU accumulate more than a decade of expertise in computer vision.

For their next vehicle, Pomerleau and Jochem plan to blend Ralph and Alvinn, which steers using brainlike "neural network" software. The video camera's field of view will be widened, and Alvinn will deal with things that can befuddle Ralph, such as freeway off-ramps and intersections on curves.

My ride with Ralph ends at the Topeka (Kan.) airport, where I pick up a rental car. Since I have time to kill, I follow Ralph westward, then speed ahead to take a picture of the minivan passing by. As I pull onto the shoulder, a state trooper follows. "I'll be with you in a second," I say. He watches me take the photo and asks, "Is that why you were doing 85?" I tell him about Ralph, and he's so fascinated that he lets me off with a warning.

Progress Toward a Driverless Car


COST $1 million $500,000 $30,000

COMPUTERS iWarp supercomputer 3 Sun workstations Sparc LX portable

and 3 Sun and 2 micro- workstation and 1

workstations computers microcomputer

TOP SPEED 1.5 mph 70 mph 91 mph*

LONGEST TRIP 2 miles 90 miles 644 miles

TOTAL MILEAGE 150 miles 1,000 miles 5,800 miles

*Test track


Before it's here, it's on the Bloomberg Terminal.