Here is the diolague (the script version) from a deleted scene that explains how they knew it was Paulie :

SONNY :"All right, Mikey...who do we have
to hit, Clemenza or Paulie?"

MICHAEL : "What?"

SONNY : "One of them fingered the old man."

Michael didn't realize that the men waiting outside were on
trial for their lives.

MICHAEL : "Clemenza? No, I don't believe it."

SONNY : "You're right, kid, Clemenza is okay.
It was Paulie."

MICHAEL : "How can you be sure?"

SONNY : "On the three days Paulie was sick this month, he got calls from a payphone across from the old man's
building. We got people in the phone company."
(he shrugs) "Thank God it was Paulie...we'll need Clemenza bad."


Don Cardi