I think it's a self-worth thing, eyes being the "window to the soul" and all is truer than you know. A contented, happy person will never have an issue making casual eye contact with someone else. When your every waking moment is spent beating your meat to the most obscene pornography possible and your social circle is filled with fellow degenerates, that becomes your routine, your habit, your existence, you. They do not want to be truly seen or truly known because the truth is horrifying and on some unconscious level they know this.