The amount of mass-scale deconstruction and reconstruction necessary to make this happen would be insane. Practically every public establishment in the country would need to undergo (what many would consider to be) an unnecessary change, coming out of their own pockets. Women and men do things in their respective bathrooms that the opposite gender probably doesn't care to see.

I don't think many females want to experience walking into a gender-neutral restroom at a bar, where some douchebag at a urinal turns around and waves his **** at her. I doubt many men want to experience bloody pads festering around. Separation for protection and privacy needs is a good thing.

Everyone who is "weird" in one way or another has to have thicker skin than the average person. For example, ugly people experience a lot of harassment. We have collectively come to the conclusion that it's pretty damn rude to point and laugh at someone if we think they're ugly, thus it's socially unacceptable. I think it's fair to say that transgender people are becoming a lot more accepted in the First World. It's not socially acceptable to harass transgender people. That doesn't mean that no one is allowed to do it or that no one will do it. There are people who will always harass those who are transgender, ugly, handicapped, African American in a predominantly Caucasian community, etc.