Not just that, but people usually forget about after those families leave the White House.
The attention is all about you, and you have people at the WH meeting your every need...but afterwards, not anymore.
I don't know about meeting their every needs, but the Secret Service continues to protect past presidents and their spouses (except when the spouse re-marries), children and grandchildren of current and former presidents until age 16,
etc.
