In both cases people's fears and concerns will need to be addressed but I don't think we can compromise of the rights of American citizens to live and worship where they please as long as they obey the law like anyone else.
I agree with you about not compromising the Constitution or the rights of the people to worship whatever and wherever they want. I just think it's in poor taste to build a Mosque at Ground Zero. That's all. It's really a propriety issue to me, rather than a Constitutional one.
Can they build a Mosque at Ground Zero? Of course, it's their right to do so.
Should they build a Mosque at Ground Zero? Nope, they shouldn't.