IRS agents for sure; they said it to him at the end.

I think that the asylum doctor was doing Hystorectomies on those women to take out the "Girl parts", which were making them act out. It's like that show "the Knick"; experimental surgery. Weird shit had to happen for us to evolve 100 years ago. Thank god we live in a civilized society today, where the only problems are ebola, rampant murder, etc. Evolution works.