When Sonny, Clemenza, etc are all holed up together, and Michael goes to visit Kay...Sonny has guys sent with him even thought "Solozzo knows he's a civilian."

Right after Michael leaves Kay, he goes to the hospital to visit Vito, and notices the guards are gone. He has to enlist Enzo to help until the Cavalry arrives.

Question - where are the guys that Sonny sent to follow Michael? Shouldn't they have been available to guard Vito since they would have followed Michael to the hospital?