I think that the novel says that Vito was always more formal and respectful towards Tom than with his natural sons.