Beside the Godfather movies, the quintessential mob movies are Casino & Goodfellas. Donnie Brasco is also outstanding. Ironically the last 3 movies I mentioned are all based on true stories, making them even more compelling. Although it's not 'mob,' watch "Heat" with Al Pacino & DeNiro. It's a very good criminal movie.
Honorable mention: Untouchables