I believe in a draft of the script Pentangeli talks to the Rosato brothers and basically says something to the effect of: "You drove Pete Clemenza to his grave, you and your brother", the indication being that they were screwing running New York up, and Clemenza was having all the pressure of dealing with it. Basically remember in the Sopranos how Silvio went to the hospital because it was too much pressure being the boss when Tony was in the coma or whatever? I'm guessing it was like that with Clemenza, but all the pressure (to say nothing of his weight, since he was pretty clearly obese) gave him a massive heart attack and he died.