For years, clerks at American Express Co. laboriously typed the handwritten numbers on every charge slip into data-processing equipment. Then, six months ago, the financial-services giant finished installing optical character readers that can decipher and process 60% of the 900,000 slips that each day pass through AmEx' two processing centers, in Phoenix and in Fort Lauderdale, Fla. The remainder -- which even humans have trouble figuring out -- are handled by clerks. The system, installed by trw Inc., cost more than $10 million, but it is expected to pay for itself within four years. Says Cliff Dodd, an AmEx senior vice-president in Phoenix: "It is critically important to us."
As pleased as AmEx may be with its state-of-the art equipment, computer-aided handwriting recognition is a technology that is just learning to crawl. A variety of writing styles, combined with sloppy or incomplete penmanship, continues to stymie scientists' efforts to devise machines that can decode every scrawl. "It's really one of the more difficult problems in computers," says Gregory K. Myers, a manager involved with sri International's research into the area. "People don't appreciate just how difficult this is."
HUGE SAVINGS. If they can advance the technology, however, a wide range of companies -- from AT&T to Hughes Missiles Systems -- expects to reach a huge new market. Banks, insurance companies, and government agencies are sure to be interested in handling their paperwork more efficiently. The technology is also important for pen-based computers, which have to decipher words written with an electronic stylus instead of a keyboard. "The market could be in the billions," says Charles L. Wilson, a manager at the Commerce Dept.'s National Institute of Standards & Technology, which is conducting research in the field.
Already, the Internal Revenue Service uses a system that reads handwritten 1040EZ forms. And in 1994, the IRS plans to award a $1 billion contract for technology that will enable it to scan all tax forms by computer by the year 2000. But the biggest potential customer of the technology is the U.S. Postal Service, which handles 555 million pieces of mail a day, 20% of it with handwritten addresses. The Postal Service foresees huge savings: It costs $3 for automatic processing of 1,000 letters, compared with $35 by hand.
Today's most advanced commercial systems, like the one at AmEx, are best at reading legible handwritten numbers and letters in predefined forms -- such as charge slips. That gives the computer an edge: It knows exactly where to look. Coping with a handwritten address is a bigger challenge. Though the Postal Service has equipment that can read most typed or printed envelopes, the systems are often stumped by handwritten street names, numbers, and Zip Codes. And when they can decipher the characters, they work at the painfully slow rate of about one address per second. The Postal Service wants to reach the speed of 12 complete addresses per second.
ZIP CODE. Making computers into speed readers involves a variety of technological strategies. At the State University of New York at Buffalo, researchers are working on computer programs that place a character in context. Given an address in Washington, D.C., the machine might have trouble if the first two zeroes of the Zip Code 20005 are looped together, making it hard to tell if they should be 00 or 06. But it may see that the word before the Zip Code begins with a "D," and recognize that this designates the state. Using a data base of addresses and Zip Codes, the system would find that there are no codes with a 6 in the middle position in the District of Columbia or states beginning with D. Hence, it concludes the number is 00.
So far, this system can read an address and Zip Code only 30% of the time, though it can decipher the Zip Code alone 75% of the time. The Postal Service, which is helping fund the work, might begin implementing the technology once the success rate for addresses gets up to 50%. Sargur N. Srihari, a SUNY Buffalo computer science professor, expects to hit 50% in the next two years.
At AT&T Bell Laboratories, nearly two dozen researchers are trying a different technique, "neural networks": a series of small computer programs, or algorithms, that work together to try to mimic the brain. In the first stage, the computer is shown different characters and told what they are. The network then sets up parameters that define those characters. To read addresses, it breaks letters or digits into parts (diagram). The number "6" would be broken down into horizontal, vertical, and diagonal strokes. If these components fall within parameters the system has learned, it concludes the number is 6.
One benefit of the neural-network approach is that the system keeps learning. It continually adds new variations to its store of parameters for each letter or numeral. For example, it quickly learns to recognize a European-style numeral 7 even though it was never taught that a 7 might contain a horizontal slash. "We have a long way to go before we approach the performance of a human being," says Bell Labs researcher Charles E. Stenard, "but we're light-years ahead of where we were a year or two ago." Indeed, the lab is working to apply the technology to a related problem: deciphering license plates on cars whizzing by at high speeds. That would enable the police to find a particular car by checking the plates on all passing cars. Stenard says highway agencies are interested in testing the system.
PEN PALL. At Hughes, engineers have gone far afield to solve the handwriting challenge -- to the pattern-recognition technology smart bombs use to find targets. Instead of trying to isolate and recognize individual characters, the system is designed to analyze and identify larger patterns, shapes, lines, or combinations of lines. That helps it ignore scratched-out words, for example, and makes it more accurate than existing systems that try to recognize individual letters, says Carol D. Campbell, who heads Hughes' efforts to adapt military technologies for commercial markets.
The challenge of decoding human handwriting is important for new pen-based computers, too, but these machines have an advantage. The computer is "watching" as the person writes -- so it can pick up clues from the sequence of the strokes and identify where the writer lifts the stylus between words. And pen computers don't have to be as speedy as mail-sorting machines. Still, they haven't yet been able to master reading even careful penmanship.
Handwriting recognition machines may never be able to decipher every doctor's prescription. But if researchers maintain their current pace of progress, sometime later this decade, perhaps, people may be able to buy the high-tech equivalent of pharmacists -- computers that can read all but the most illegible scrawls accurately and quickly.