Jame Cagney - His movies may not be as high in quality as The Godfather, but they are still great films. He was there since the beggining of the genre, with The Public Enemy, G-Men, and The Roaring Twenties.
Al Pacino - Has played the most memorable characters in film history(IMO). Most Notable, Michael Corleone, Frank Serpico, Lefty Ruggiero, Tony Montana, and Sonny in Dog Day Afternoon.

I also have to mention some of the other greats, Robert DeNiro, Edward G. Robinson, and Joe Pesci.