High Anxiety for Banks

The refi craze could backfire as rock-bottom rates cut into yields on mortgages -- and profits

After buying his Austin (Tex.) house in 1984, Patrick E. Cassidy rode along for more than a decade with 10%-plus mortgages. But as rates began their sharp descent in recent years -- and banks made the mortgage process much easier -- refinancing has become an almost annual ritual for the Southwest Texas State University professor: He swapped his old mortgage for a 7% note in 2001, then pulled the trigger again on a 5.75% loan last fall. And with 15-year rates now below 5%, Cassidy is ready to refi again. "Refinancing is not even (worth) a discussion," he shrugs. "We just look at the rates and say, 'O.K."'

Cassidy has plenty of company: With rates at their lowest levels in nearly four decades, refinancing has become the great American pastime. And so far, the trend has been just as good for the banking industry, which has raked in record profits from the housing boom. But given how far and how fast interest rates have come crashing down, the good times may be winding down for lenders. While bankers still earn fat origination fees from the "points" they charge borrowers, those profits are increasingly offset by the much smaller returns that banks are getting from lower-yielding mortgages in their investment portfolios.

Moreover, banks and other mortgage players have paid handsomely in recent years for the right to process homeowners' mortgage payments. Now, they are seeing that servicing income melt as borrowers like Cassidy refinance. Suddenly, this one-two punch of smaller loan yields and servicing losses is creating an unexpected squeeze in bank profits. "We've never been where we are now in terms or mortgage [rates] and the ease of refinancing," laments one top executive of a major U.S. bank. "It's like flying a C-130 in the middle of a hurricane. It's very dark, volatile, and dangerous right now."

The mortgage morass is the most pronounced example of how the ongoing plunge in rates is starting to take a toll on banks. Until recently, the decline has been orderly. And banks were always able to maintain a wide gap between what they paid to borrow money and what they could charge for mortgages. But with banks now paying 1% or less for most deposits, that's squeezing those margins. "You just can't go lower than what we are" paying to borrow, Steve Smith, chief financial officer of Palo Alto (Calif.)-based Greater Bay Bancorp., recently told analysts.

The squeeze is starting to pinch profits. According to Sharon Haas, an analyst at debt-rating agency Fitch Ratings, the spread between lending and borrowing rates for the 30 largest banks has fallen from 4.13% to 3.84% since March, 2002. Haas warns that could shrink another quarter point in coming months.

That's not the only hit from falling rates. Refi madness is also causing losses at banks that have built big businesses in processing mortgage payments. With refinancings coming faster than they'd expected, those that paid hefty premiums to acquire the servicing rights to loans are having to take write-downs as the loans they bought disappear when they're refinanced. One such victim: HomeSide Lending (NAB ) Inc., which sold its servicing business to Washington Mutual (WM ) Inc. last year after taking write-downs of more than $1 billion. "There's a saying among mortgage bankers: 'Defaults hurt, but prepayments kill,"' says San Francisco consultant Michael Bykhovsky.

So given all the headaches from falling rates, will a pickup in rates prove a panacea? Perhaps not. That would likely just create a different set of problems for the industry. For one, with seemingly every bank plunging headlong into mortgages, the industry will incur heavy restructuring charges as many players scale back their businesses when the boom fades. Another concern: In their quest to maintain profit growth until corporate lending rebounds, banks that traditionally held Treasuries during downturns have instead loaded up on mortgages, which yield roughly a percentage point more. Among the 50 top banks, mortgage-backed securities now constitute 62% of their investment portfolios, vs. 47% in 1994, according to U.S. Bancorp (USB ) Piper Jaffray analyst Andrew B. Collins.

Problem is, by betting big on the more volatile mortgage market, many banks could see a sharp drop in the value of their mortgage holdings when the economy strengthens and rates rise. Mortgages that banks expected, based on recent trends, to hold three or four years before they were refinanced away could suddenly turn into 30-year maturities if homeowners suddenly decide to hold that loan until they pay it off. That could leave banks sitting on low-yielding paper at a time of rising rates. "If we have a sharp pickup in rates, you'd see banks suffering pain in their investment portfolios," says Arthur Frank, director of mortgage-backed research for Nomura Securities (NMR ) International Inc.

Whether banks' increased exposure to mortgages remains a minor problem or develops into a full-blown crisis depends on how well they are hedged against a sharp rise in rates. The issue "is getting lots of supervisory attention -- banks are being told to get their hedges in place now, while times are good," says Nancy A. Wentzler, deputy comptroller for global banking at the Office of the Comptroller of the Currency. Indeed, how banks handle the challenges ahead may well determine whether the mortgage business becomes a victim of its own success.

By Dean Foust in Atlanta, with Brian Grow in Orlando, Mike McNamee in Washington, and bureau reports

    Before it's here, it's on the Bloomberg Terminal.