It's in Philadelphia, it looks like a Castle, it's no longer in use. They have a guided tour that goes through certain parts of it but I'm almost positive you are allowed to roam most anywhere.

"You gave your word, I never gave mine"
http://s2.gladiatus.us/game/c.php?uid=88380