Frankie being nostalgic with Tom in military compound said that the Corleones was once like a Roman Empire. I suppose he meant that the Corleones going down and down. When do you think the glorious period that he was talking about.
Frankie judgement was wrong then because in GF III the family is getting richer and getting more powerful