I don't think Rocco would have betrayed the family, as he felt obligated that they were treating him very well compared to his old job as a policeman. Rocco was blamed for his failure as security chief of corleone compound, and this led him to being a suspect in the attempted assasination attempt of Michael. I don't think a traitor would go on a difficult, but not impossible mission.