When Anthony was called away, Fredo's whole demeanor changed;voice, facial expression, body language .. Not the kind of change of disappointment, more like resignation. I think he knew. Anthony's trip was part of the plan to get Neri and Fredo alone on the lake, so Al was prepared.