I think Mike knew before the boathouse scene that he was going to have to get rid of Fredo. He only went in their to get information about Roth and the investigation IMO.

"Fredo... dont ever take sides against the family again... EVER!" - I think Mike really meant it and was letting it be known that he would pay for that.

In Mikes mind... Fredo had betrayed Mike more than just the shooting in his bedroom.... w/ Moe Green etc...

It was him or Fredo. He chose to live. Thats about all their is to it.