Vito is supposed to be such a great family man:
"VITO CORLEONE (to Johnny, after glancing to see Sonny enter)
You spend time with your family?
JOHNNY
Sure I do.
VITO CORLEONE (to Johnny, but toward and about Sonny)
Good. 'Cause a man who doesn't spend time with his family can never be a real man.
Then what is he doing running to Italian shows with Genco, and leaving his wife home alone with his son in that dumpy Ralph Kramden apartment?