I think that some of you are envisioning Vito's construction of his crime family apparatus and its operation as an affair that took place under the noses of his family. Why think that Roth or Ola was ever in proximity to Sonny or Fredo as kids? Sonny was at least 16 before Vito admitted him to any exposure to the business and that was only because he claimed to have witnessed Vito's murder of Fanucci. Even then, as the novel states, his exposure was quite limited until the Maranzano War.
Ola knew Tom. Tom and Sonny were the same age. I don't think that as children or even as teens either Tom or Sonny knew Roth/Ola but I do think each of them would have run across Roth/Ola say in the years between 1934~1941 or so. Sonny would have been a bodyguard/driver for his father and Tom got to sit in on some meetings. I don't think it's too far out there that Ola would have remembered Tom and Fredo too-once he joined the olive oil business.