It was racists and conservatives/libertarians in both parties who were the problem. In the South most of these people were Democrats because of history going back to the Civil War. Northern Democrats i.e. FDR attempted to paper over these differences.
As more black people got the right to vote and became formal citizens many of the Southern white Democrats switched parties, a gradual process which started with Goldwater's opposition to the 1964 Civil Rights Act and sped up with Nixon's Southern strategy. Post 50s and 60s changes the Democratic Party could no longer hold together a coalition which included Southern segregationists, Midwest farmers, and East Coast Liberals among others.
Two good books exploring this phenomenon are John Hope Franklin's "Mirror to America" and Jeremy Mayer's "Running on Race".
"When the snows fall and the white winds blow, the lone wolf dies but the pack survives."
Winter is Coming
Now this is the Law of the Jungle—as old and as true as the sky; And the wolf that shall keep it may prosper, but the wolf that shall break it must die.
As the creeper that girdles the tree-trunk, the Law runneth forward and back; For the strength of the Pack is the Wolf, and the strength of the Wolf is the Pack.
| 
