Like Michael, Vito was absolutely sure of Tom's loyalty for all those reasons. Sons are sons, but Sonny was the only one of them that Vito employed at all initially in helping with the business. Tom actually asked to work for Vito and Vito obliged him by involving him more and more in the business. I don't think Sonny was jealous. Fredo just silked about it or may have not been intelligent enought o see Tom's closeness to Vito. Michael could have been authentically jealous.
I agree Fedo was probably not bright enough to see it, but Sonny showed it...
Sonny - That's easy for you to say, he's not your father.
Tom - I'm as much a son to him as you or Mike. (Note he omits Fredo....no respect!)
and this
Sonny - Pop had Genco look what I got.
Tom - (looks hurt)
Sonny - I'm sorry, look Mom made a little dinner....
and this
Michael (to Tom) - You talked to MY father about MY future!