It happens both ways. When says something "offensive" the problem is that a subject is taboo for a reason, to not involve personal attachment to the subject. Thus is offended because now the discussion is focused on how close people are to each other which makes for awkwardness because feelings of hate/dislike may surface.

When says something "offensive", the problem is that the implications on the distance between people was not considered. Thus an valuer may attach a hate or dislike label to the person for offending values.