I think, PrimoPaisan made some valid points. Even Vito kept an eye on his family. The novel tells us that even though he didn't talk to Fredo, Vito insisted on being informed about his behaviour. He was regularly informed about Carlo's actions. And he seperated Clemenza and Tessio spatially after he had risen to power. He handled Luca carefully, like a valued, but dangerous dog. Luca was not a real friend to Vito, even though he says so at the wedding party. To Michael, Neri seems to be the only friend.