Republicans have complained that hiring more IRS agents means that more middle-class Americans will end up being audited. The IRS has instead said that they're struggling to go after the wealthiest taxpayers because it means a protracted legal battle and court fees.Â