I have thought, at times, that Michael might have been ready to give his brother a pass. But Fredo's outburst in the boat house showed the depths of his resentment of Michael. I think Michael concluded at that point that Fredo would alays be a danger to him.
Keep in mind that Fredo knew the Feds had Pentangeli. And, after the outburst, he tells Michael that "the Senate lawyer, Questadt, belongs to Roth." That was info Michael didn't have. Had Fredo wanted to get back into Michael's good graces, he could have told him that long before Michael perjured himself five times. That tells me (as it must have told Michael) that, even after his perfidy was exponsed in Havana, Fredo was still unrepentant.
Sums up what I was thinking. When Micheal says "you're still my brother" and when he tells Tom to tell Fredo "not to be scared" makes me think he wasn't decided that he would kill Fredo. Something convinced him, probably the realisation that Fredo knew more than he let on and that he was in a position to help Micheal but didn't.
