IRS staff cuts mean fewer audits of the wealthy

A version of this article first appeared in CNBC’s Inside Wealth newsletter with Robert Frank, a weekly guide to the high-net-worth investor and consumer. Sign up to receive future editions, straight to your inbox.

Wesley Stanovsek was a dream hire for the IRS in 2024.

With $80 billion in new funding from Congress, the IRS went shopping for young, tech savvy accountants and engineers who could deconstruct the complex returns of the wealthy and private companies. Stanovsek, based in Columbus, Ohio, specialized in S-corps, trusts and partnerships before getting hired by the IRS’s High Wealth division.

In February he was fired, along with other IRS agents who were considered “probationary” since they had been there less than a year. Stanovsek was working on three so-called “enterprise” cases at the time – two involving partnerships and one involving a wealthy sports team owner, totaling millions of dollars of potential additional taxes.

When he left, the cases were dropped due to lack of staff.

“They’ll most likely be closed with no change,” said Stanovsek.

With the IRS expected to lose about a third of its staff after firings and buyouts, wealthy taxpayers and attorneys are struggling to navigate the new regime. The upside for high earners is clear: Fewer agents means fewer audits and reviews.

Under the Biden administration, the IRS made a concerted effort to target the wealthy, aiming to double audit rates for those making more than $10 million a year and launching a campaign to investigate private-jet owners. Now, especially with the potential dismantling of the Department of Justice Tax Division, the tax enforcement ranks are rapidly dwindling.

“The agency is like a zombie right now,” said Kathleen Pakenham, partner at Vinson & Elkins, who handles corporate and high-net-worth tax cases. “There is no brain in charge of what’s happening on the ground.”

Attorneys say many of their audits have gone dark. With the statute of limitations on an assessment typically limited to three years after the taxes are filed, many are expected to expire without attention.

Some wealthy taxpayers are asking their attorneys and accountants whether they even need to bother filing returns. The answer: an emphatic yes.

Since the IRS has historical data on every taxpayer who has ever filed, a missing filing in one year would immediately raise an audit flag by the IRS’s automated system.

More likely than an all-out boycott, say attorneys, is a new era in which taxpayers and their accountants push the envelope with aggressive tax-planning techniques that may escape the review of an understaffed agency. Pakenham said IRS budget cuts in 1999 and 2000 led to wave of esoteric tax structures that more than a decade later proved to be illegal.

“This is exactly the kind of environment in which tax shelters are devised and sold and more widely implemented,” she said.

A Yale Budget Lab study found that if the IRS staff cuts endure for the next 10 years, tax revenue will fall at least $160 billion over the next decade. Other reports estimate the lost revenue at over $500 billion this year alone. Former IRS agents and tax attorneys say the specific nature of the recent cuts will have an outsized impact on collections from the wealthy – and therefore a larger impact on overall revenue collecton.

Jack McCumber was a real estate and business appraiser with an information systems background before getting hired by the IRS for its Large Business and International unit, which audits high-net-worth individuals and companies with more than $10 million in assets. Before he was fired, he was working on cases totaling more than $150 million in taxes under question. Some related to syndicated conservation easements, which the IRS included on its 2024 list of “Dirty Dozen” tax schemes.

McCumber said the majority of his LB&I team was new hires at the agency, who were swiftly fired. The group had one of the highest returns-on-investment at the IRS: Every 33 cents the group spent on enforcement resulted in $100 of added revenue, he said.

I don’t think there will be enough people to take over all those projects,” he said.

At the same time IRS audits and enforcement fall, accountants and tax lawyers say it’s likely the agency will have less capacity to provide decisions or paperwork that can solve a tax problem.

Robert Romashko, a partner at Husch Blackwell, said he has one audit case open in which the IRS claims the taxpayer owes up to $8 million. He requested an appeal in December, but an appeals officer has yet to be assigned to the case.

“Typically that would have happened by now,” he said. “I attribute that to a lack of personnel.”

The taxpayer is left in limbo. If the case isn’t resolved by the end of the year, the statute of limitations expires.

Another of Romashko’s clients is trying to sell a business that includes a property with a legacy tax lien. The client is trying to pay off the lien plus interest so he can sell the business but can’t get anyone at the IRS to respond, which is holding up the sale.

“It’s been a nightmare,” Romashko said. “It should be easy.”

IRS personnel are also critical for helping with delayed filings. Many wealthy investors have overseas bank accounts or investments they’re required to disclose but often take months to receive. Typically, an agent can help resolve the delay without a penalty. But an IRS with fewer people allows its automated systems to take over and possibly impose a tax lien on a client’s assets, Romashko said.

“There are areas where we need the service, otherwise things disappear into a black hole,” he said.
Treasury Secretary Scott Bessent, who oversees the IRS, told CNBC in March that cost-cutting at the agency won’t impact collections. In fact, he said AI and other new technologies will allow the agency to be more efficient and even do a better job at collections and service.

“I have three priorities with the IRS,” Bessent said at the time. “Collections, privacy and customer service — in that order. So, there’s nothing, nothing I’m gonna do to hurt the collections over time. We are in the midst of this great AI boom, and, you know, I think increasing headcount now would be just the wrong time as the private corporations are moving into AI. I can’t think of a better application for AI than auditing tax returns.”

Attorneys say they’ve already had several encounters with the IRS’s AI agents, and the results are mixed. AI is highly effective at selecting the types of returns that might contain certain kinds of abuses or strategies.

“Normally for an auditor, it’s like they open the closet door and start digging around and asking questions,” Pakenham said. “Now it’s like they have X-ray vision. They already knew some of the things in the closet.”

Once the issues are discovered, however, attorneys said it takes a highly skilled auditor with years of experience to ask the right follow-up questions and make judgements.

“With a human, you can reason and explain,” Romashko said. “If AI comes to the wrong conclusion, it’s much harder to shake it. You can’t talk to a computer.”