Personally, I wish the Hyman Rothstein scene was left in. One of my favorite aspects of PtII is the interplay between the two stories, past and present. I think much of the sadness of the Michael segments comes from the contrast to the young Vito segments. It's quite moving, after Michael and Roth's deadly battle in Cuba, to show Young Roth's interaction with Young Vito. Just to think that this kid Vito takes under his wing would, some 35 years later, be in a life-or-death power struggle with Vito's son.
Just imagine how (possibley) more powerful PtII would be if the plot had stayed with Clemenza turning on the family and killing himself in the end, instead of Pentangelli! Gotta love Pentangelli, though...
Plus, I agree with Turnbull that Clemenza's transformation from big-shot (At least in his head) to side-kick is really entertaining.