How Bad Studies Get Turned Into Bad Policies

Economists cannot design research projects that decisively answer important policy questions. They have to rely on the limited knowledge that can be extracted from the effects of government policies and other events. Yet politicians are all too quick to grasp preliminary results of these studies as justification for regulations and other interventions that sometimes do damage to the economy.

A good example is a 1992 Federal Reserve Bank of Boston analysis of discrimination against applicants for mortgage loans. The senior author was Alicia Munnell, a respected economist who was then in charge of research at the Boston Fed and recently was nominated by President Clinton to fill a vacancy on the Fed's board. After reviewing what happened to some 4,500 loan applications in 1990 by banks in the Boston area, the authors claimed to have found evidence of extensive discrimination against black and Hispanic applicants.

The publicity that this study attracted led the Comptroller of the Currency to closely monitor banks for evidence of discrimination in their lending practices and to investigations of banks by the Justice Dept. and the Massachusetts Attorney General. Such scrutiny and political pressure resulted in legal actions against a few banks. One involved a consent decree last August with a Maryland bank for supposedly "shunning" loans in urban areas. Another involved settlement of a case against Chicago-based Northern Trust Corp., which alleged the bank gave greater assistance to white applicants who were attempting to qualify for mortgage loans.

Such political reactions would be justified if the Boston Fed found convincing evidence of bank discrimination. But the study--despite being lauded as conclusive--contains many omissions and data errors. An economist at the Federal Deposit Insurance Corp. took a close look at a sample of the actual lending records used by the Fed and concluded that the data are so inaccurate (owing, in part, to errors in copying), that it is impossible to determine whether or not banks had discriminated against blacks and Hispanics. Another analysis found that the Boston Fed's results are dominated by two banks that had relatively high rejection rates of minorities only because they specialized in soliciting loans from marginally qualified minority applicants.

Some of the evidence found by the Boston Fed contradicts their claim of discrimination against minorities. For example, average default rates found in this study were about the same on loans in census tracts with a large percentage of blacks and Hispanics as in predominantly white tracts. Yet if the banks had been discriminating against minority applicants, default rates on loans to minorities should have been lower than on loans to whites, since banks discriminate in part by accepting minority applicants only with exceptionally good credit histories and employment records. They would reject marginally qualified minority applicants while accepting marginal white applicants. The implications of the theory of discrimination for default rates and other behavior are spelled out in my Nobel prize acceptance lecture entitled "The Economic Way of Looking at Behavior," published in 1993.

WAGES OF ERROR. I am not claiming that all the additional studies of bank lending practices concluded that banks do not discriminate, nor am I suggesting these studies "prove" banks generally do not discriminate. But the additional analyses do show the difficulties in reaching recommendations on controversial policy questions by generalizing from highly imperfect evidence.

These difficulties are further illustrated through recent discussions of the employment effects of minimum wages. The most basic law in economics states that a rise in the cost of labor, capital, or other inputs lowers demand for that input, a law that has been verified by the experience of thousands of companies all over the world. Yet Labor Secretary Robert B. Reich cites controversial studies that claim to find rises in minimum wages have sometimes not reduced, and may even have increased, employment of low-skilled workers. My discussion of some problems with these claims appeared in BUSINESS WEEK of Mar. 6, 1995.

The effective responses to the allegations that banks discriminate and that higher minimum wages do not reduce employment should be a lesson to economists not to advocate major policy initiatives on the basis of very limited and imperfect evidence. Politicians are only too eager to promote results based on the flimsiest of evidence to attract support for policies that help voters and interest groups they favor.

Before it's here, it's on the Bloomberg Terminal.