I never made the drive myself, but it's not that far. According to THIS it should take less than 2 hours....

Edit: 3.5 hrs? In SB-time I guess. :p

NYC is about 1.5 hrs north of me, and Philly is about an hour due west of me -- even if you detoured to the Jersey Shore you could make it in less than 3.5 hrs