To add another nuance from that scene in the novel:
After Connie's wedding, Paulie and Clemenza drive Kay back to her hotel. "He [Michael] had led her to believe that he was an alian in his father's world. Now Clemenza was assuring her, in his wheezing gutteral voice that the 'old man' thought Michael was the best of his sons, the one who would surely inherit the family business."

Michael obviously didn't tell that to Clem, so Vito must have--already deciding on his heir.