Tom knew Fredo was harmless and was basically set up by Roth. Plus Fredo was basically a brother to him. However by the end of the film Tom realized how psychotic mike had become so im sure he was careful not to say anything that might trigger him. Also Tom probably didnt think mike would actually stoop to that level