Well, this may be a minority opinion, but I think Michael believed he was trying to be fair to Fredo:
--It was Vito's decision, not Michael's, that Michael be head of the family. A no-brainer anyway, since Fredo was a no-brainer.
--Fredo had embarrassed the family in Vegas by acting in a "degenerate" (2-at-a-time) way that upset Vito, and by permitting Moe Green to slap him around in public. He made the family look weak.
--He compounded the Vegas mistakes by taking Moe's side in the dispute over buying Moe out. If Sonny's remarks at the Sollozzo meeting exposed a ch**nk in the family's armor, so Fredo's remarks in Vegas made it look like Michael really wasn't in charge. A case could be made that Fredo's suggestion that Michael wasn't the last word in that dispute might have led Moe to hold out, resulting in his murder. Fredo certainly made Michael look bad.
--Fredo married a trampy slut--a drunk, a flirt, and a bigot to boot ("Never marry a wop...") who, among other things, made an embarrassing scene in front of myriad important guests at Anthony's party--an important event in Michael's "legitimization."

Despite all of this, Michael supported Fredo and let him live on the compound. Yes, he gave him "Mickey Mouse" jobs, but probably they were the only kinds of jobs a dimwit like Fredo could handle. Probably those "Mickey Mouse" jobs kept Fredo from getting into serious trouble on his own.

And how did he reward his brother's perseverence? By betraying him to Roth and Ola.

Fredo resented Michael because he was "passed over" for the Donship--an expectation that was totally ridiculous in his case. It wasn't Michael's fault that Fredo wasn't qualified to run a two-car funeral, much less a the country's top Mafia family. His death was a consequence of an act of betrayal of his brother that he chose to commit.

Michael was, definitely, a cold-hearted bastard. And killing his own brother was a reprehensible, revolting, immoral act. Michael was a product of the environment he chose. His behavior, and the disasters that befell him, were consequences of the choices he made. Rough justice.