Other than the horse and Fanucci, the films didn't show us much of Vito's ruthlessness. But the novel did--attest the murder of the Capone gunmen. I don't see any substantive difference between Vito and Michael in the ruthlessness department: it comes with striving to be a Mafia pezzanovante.

As for Carlo, in a deleted scene, Vito admits to Michael that his failure to act on Sonny's and Appolonia's murders were "weaknesses." But there were pragmatic reasons for not whacking Carlo. As has been pointed out here, it would have signaled Barzini and the others that the Corleones knew who set up Sonny--and who was behind it. That would have been fatal to the Great Massacre of 1955. Another: if Michael had had Carlo whacked before Vito died, Connie would have gone running to her father, driving a wedge between Vito and Michael and undermining Michael's authority.