One of the things that makes Michael reprehensible is the choices he made, consciously and knowingly. When he chose to become the Don, he chose a way of life that made his killing of his own brother a necessity, to his way of thinking.

Fredo had betrayed Michael to his enemy, and endangered not only Michael's life, but Kay's. As a brother, Michael of course could have given Fredo a pass--punished him, perhaps, but spared his life. But, as Don, Michael believed that, if he showed any sign of "weakness"--even toward his own brother--others might get the idea that he had a "soft spot" that could be exploited. Tessio thought him weak in the face of Barzini's provocations, and betrayed him. After the Tahoe shooting, Michael could not rule out the possibility that Rocco and Neri--who, earlier in the day, he described to Geary as men he trusted with his life--were part of the conspiracy. Call it paranoia, but it comes with choosing to be the Don: anyone, no matter how close to you, could become your enemy.

(N.B.: Binnie, you posted a good question, but it belongs in the Trilogy section because killing Fredo wasn't in the novel.)