Simply put it was easier to keep the grip on a smaller area/city with almost no competition than having to knock shoulders with other 4 crime families in one of the world’s largest cities. Shaking down “cow towns” along the rust belt and the Midwest wasn’t that much of a task for them.

Not to mention that the whole small town America vibe made local mafiosi the top dogs right off the bat.
Smaller/rural centers don’t get the same scrutiny larger cities and metropolitan areas get. Public officials are less likely to get on the feds’ radar, thus they’re easier to grease. Omertà is more stable when literally everyone can trace your family tree and everybody knows everyone.

In NYC (or other large cities such as Chicago) they had/have to navigate through a much more crowded underworld and constantly changing politics and demographics.

The ethnic and cultural canvas of places like Youngstown and Upstate NY, eastern PA, etc... haven’t changed that much in the last 50 years.

Ironically, what brought down many of these smaller families was their involvement in larger, nation-wide rackets with bigger coastal families (Unions, Vegas,...). If they’d have kept it low key and local, chances were they’d still be around today.
All this plus the fact that the bosses run these families as their own private clubs: very little room was given to other up-and-coming guys so you had no new blood coming up. Small families were run like close-knit kingdoms while larger ones were run like proper governments.