GFII provides us with a paucity of information about the underworld at the time. Did it consist of the Corleones, Roth, and whatever family (if any) represented by Ola? For sure, both Ola and Roth referenced some OC people and one gang ("The National will go to the Lakeville Road boys. The Capri to the Corleone Family. The Sevilla Biltmore, also, but Eddie Levine of Newport will bring in the Pennino Brothers -- Dino and Eddie - for a piece, and also to handle the actual casino operation"), but it's a challenge to believe that NY's OC population consisted solely of the Corleones and Rosatos.

Had Michael been murdered, Pentangeli was already in place. Who besides the Rosatos would have challenged him for NY dominance (I don't think the Rosatos could have seriously challenged him).