Don Altobello was the consigliere of Rico Tattaglia from 1955 until 1961,at which time he became Boss of that family.He and Vito could have become acquainted at any time on the streets of New York.
I don't think that their respective ranks would make much difference. For all we know,they could have met when Vito was working for Abbandando's Groceria.Two guys in different Families who happened to become friends.
Mike refers to the long term friendship with his Family,so who knows?