Anthony was supposed to go fishing with Uncle Fredo when Connie took him out of the boat. Although he may not have understood the implication of the boat returning without Fredo, he certainly could have put the pieces together when he was older.

I don't think Mary knew about Fredo's murder. She had a blind love for Michael, the type of love that only a very spoiled and sheltered daughter could have for a father.