The book simply says he disrespected Vito, and that Vito had to act or else he would've lost the respect of Clemenza & Tessio. And in this business, respect is everything.