I thought it was implied Ola left after he talked with Michael. There is a chance that they didn't even see each other at the party. Of course the plot against Michael had already been accidentaly set into motion by Fredo anyway. Fredo at the time he denied talking to Roth or Ola was just trying to get Michael off his back. Pretending to be in the dark about the hit/deal was the best thing he could have done.