When someone says they are sooner worried about A than B, it does NOT mean A is more dangerous than B. It means the likelihood of A happening, is more than that of B, though A is not very likely to happen either.

Plus, they are all ideologies. It's not like you could accomplish much by worrying about it. I personally wouldn't worry, because both of these ideologies wouldn't stand the test of logical debate. And strangely enough I have faith in common sense, and I do think the majority of people would eventually see the fault in these extremes.