The base level of Fredo's stupidity is established by, "...ah...wrong number." There's no reason to think he ever suspected anything about anything.

"All of these men were good listeners; patient men."