I don't think people hate him. He just gets made fun of for that whole 'there's a 500k bounty on me so I gotta creep around in my awesome shades' nonsense.
Plus I personally don't think he could have went so deep in a family like that for so long without crossing lines he says he didn't cross. But that's just a hunch.