I have to disagree with all of you here.

1-Vito has the police on his payroll.
2-If he hadn`t he could have a license to carry a gun in case he is caught by the police, because all his men have it(Remember Tom Hagen says to McClusky that after Michael was hit on his face). I mean. if the soldiers have a license, why not him?
3-If Vito is caught with the gun, he can say rich and powerful business men could be the target of anybody, so It is for self protection.
4-If he had to shoot the two attacker, he can say "self-defense"