He looked like a girl with his old haircut and his voice was annoyingly high pitched (pre-puberty) so that is where that came from. I even know people that heard the song and thought the artist was a girl just by the voice.
Plus, men make fun of any guy that is attracting a lot of female attention to attempt to drive attention off of him and onto them.

Howver, I do enjoy his current music as it isn't so high pitched anymore. As long as you love me is one of my favorite tunes by him.