I'd say the Genovese were bigger and more powerful, or at least equal to the Gambinos during the 60s. The thing was Vito was in prison and the three leaders in his place weren't always on the same page. I think Gambino took advantage of that, plus he was allied with Luchese, Colombo and Bruno so he held a lot of sway. I wouldn't bet that he ever dictated how things went to guys like Benny Squint and Fat Tony though.

By the time Castellano and Gigante were in power it seems the tide had turned a bit. According to Gravano, Gigante was doing a lot of manipulating of Castellano, not the other way around.