With the exception of Fredo, the others weren't his family, Danito.
As far as Fredo goes, it would be natural for a caring father to be concerned about a "slow" child and would want to be kept abreast of what was happening in his life.
Exactly SC. It's important to distinguish between Vito's family and his Family.