Given that Norma's brother did not deny the accusations but said "it wasn't like that", you almost have to wonder if whatever happened between the siblings was consensual or if one or both was/were harmed by one of the parents. Rewatching it, it seemed as if the brother wanted to say something else about Norma but held back.
Also for someone who is supposed to be a geeky weird nerd, Norman certainly has the young ladies showing interest in him.
