Ex-Billionaire on Tax Shelters: I Trusted My Lawyers. Judge: Not Good Enough

Sam Wyly

Sam Wyly arrives at U.S. District Court in Manhattan in May 2014.

Photographer: Kathy Willens/AP Photo
  • Wyly lost a related SEC fraud lawsuit in Manhattan in 2014
  • Dallas bankruptcy judge says Wyly can’t ‘hide behind others’

Former billionaire Sam Wyly may have been hoping for a better outcome on his home turf as a Texas judge weighed tax-evasion claims related to the fraud trial he lost in New York two years ago. He didn’t get it.

U.S. Bankruptcy Judge Barbara Houser in Dallas roundly rejected the 81-year-old entrepreneur’s argument that he was simply following orders from his own employees when set up a web of offshore funds that hid his assets, allowing him to make hundreds of millions of dollars in illegal profits.

“The court does not believe that the law permits Sam to hide behind others and claim not to have known what was going on around him,” Houser said Tuesday in her 459-page ruling.

The decision is the latest blow to Wyly, once a fixture in Texas high society, in a string of legal battles with the U.S. Securities and Exchange Commission and the Internal Revenue Service that threatens to wipe out the fortune he amassed over a lifetime building companies including the arts-and-craft chain Michaels Stores Inc. and Sterling Software Inc.

The IRS was seeking $1.4 billion from Sam Wyly and $834 million from his sister-in-law, with penalties and interest accounting for 80 percent of the totals, the government said in court papers filed Jan. 25. But instead of deciding on those claims as requested, Houser on Tuesday gave the parties 30 days to confer and submit “agreed amounts” on the claims to the court. Failing an agreement, each side is to submit its own proposal, Houser said.

Estate Lawyer

Faced with Tuesday’s ruling, the Wylys will probably have to strike a deal with the IRS and pay a “significant” amount of tax to resolve the case, according to Laura Zwicker, who chairs the private-client services group at the law firm Greenberg Glusker Fields Claman & Machtinger LLP in Los Angeles.

“For the tax planning community generally, it’s really a wake-up call and a further indication that the U.S. courts are not gong to allow abusive tax planning to take place,” said Zwicker, who isn’t involved in the case.

The IRS argued it was the victim of a vast fraud revealed in a 2010 SEC suit against Sam and Charles Wyly, brothers and longtime business partners. Charles died in a car crash in 2011, leaving his widow, Caroline “Dee” Wyly, caught in the fallout of the litigation.

Hidden Holdings

In 2014, a federal jury in Manhattan found the brothers had used a web of offshore trusts for 13 years to hide stock holdings and evade trading limits, allowing them to rake in $550 million in illegal profit. The verdict quickly triggered bankruptcy filings by Sam Wyly and his sister-in-law. And then the IRS fight began.

Houser held a two-week trial in January in Dallas to determine whether the Wylys defrauded the IRS. The proceeding shed light on the assets and lifestyles of the extended Wyly family, including their Dallas mansions, expansive ranch properties in the mountains of Colorado and rare artwork.

The IRS argued many of the luxuries were purchased by offshore trusts and “loaned” to the family to avoid taxes, and that property was gifted to children for the same purpose.

Lawyers, Accountants

Sam Wyly said he had relied on lawyers and accountants to set up the offshore trusts and knew few details about how they operated. During the trial, his lawyers called the arrangement “aggressive but not illegal.” Dee Wyly testified that she entrusted financial matters to her husband and signed tax returns and other documents without reading them, an argument that Houser largely accepted in her ruling Tuesday.

But the judge had little patience for Sam Wyly’s defense.

“To accept the Wylys’ explanation requires the court to be satisfied that it is appropriate for extraordinarily wealthy individuals to hire middlemen to do their bidding in order to insulate themselves from wrongdoing so that, when the fraud is ultimately exposed, they have plausible deniability,” Houser said in the ruling.

The judge, however, found that Dee Wyly hadn’t participated in any fraud.

“There is simply no persuasive evidence in the record that Dee understood how these very complicated estate planning transactions worked,” Houser said. “Dee did not have the educational background or sophistication in business and tax matters to know
if her tax returns contained any understatements of income”

Mixed Reactions

Stewart Thomas, general counsel for the Wylys, said they were pleased with the ruling on Dee, as well as the court’s rejection of an IRS gift-tax claim, but “they are surprised and disagree with the court’s fraud finding as to Sam and his brother Charles.”

The case is In re Samuel Evans Wyly, 14-35043, U.S. Bankruptcy Court, Northern District of Texas (Dallas).

Before it's here, it's on the Bloomberg Terminal. LEARN MORE