I never quite understood why this was such an insult - maybe he's indicating that Moe doesn't really have control over the situation.
A casino doesn't lose money by being unlucky. He pretty much accuses him of skimming without using the words. He did mean it as an insult to put Moe on the defensive and take control of the conversation to tell him he was going to be out of the casino business one way or the other.
Long as I remember The rain been coming down. Clouds of Mystery pouring Confusion on the ground. Good men through the ages, Trying to find the sun; And I wonder, Still I wonder, Who'll stop the rain.
