Word of mouth, the fact that Vito told Clemenza that he had lost the gun and paid him for it. Probably started with a rumor (albeit a true one) and just snowballed from there.