I think one of the catches with the introduction of Google Glass is that it's a shift in paradigms. People that are hostile towards Google Glass are uncomfortable with the lack of identification to a similar device. I bet that 99% of the people you know still refer to their smartphone as their "phone". The telephone has been around for over 100 years. People are comfortable with the concept of a phone. The phone allows them to communicate. And though the methods vary, people still communicate on their phone. Granted, one of the least common activities on today's smartphone is probably the actual phone call, but people still see it as a communications device when in reality, it's a computer. That thing we call a phone is a real, live, fully functional computer and it is capable of much more than an old Princess Phone ever dreamed of doing.
People are used to glasses being used to see things, and that's it. They're not used to glasses being used as a communications or computing device. Glasses are used to improve eyesight. This shift in paradigm is making people uneasy. There's no "oh, it's a phone!" type identification going on. The baseline is missing.
While there is a lot of FUD out there about Google Glass (for example, Google Glass in its default configuration will not allow the user to take a photo simply by winking, despite what others are screaming from roof tops, you have to add that functionality to the device), I do get that people are concerned about folks recording a moment they shouldn't be recording. That problem exists all over the place today. Where there's a phone, there's a camera. Where there's a phone, the potential of a recording device exists. Where there's a public street, the possibility of a surveillance device exists. One quick side note, I find it weird when articles talk about things such as "privacy in public spaces." When did we expect to be incognito in public?
This is where society is going to have to set etiquette standards on the proper use of Google Glass and other wearable technology. As technology evolves, an "all or nothing" approach is going to get us nowhere. If you don't like the technology, don't use it? If you're uncomfortable with the technology that's being used by someone near you, politely say to them, "would you mind taking your glasses off?", with the same approach and social grace as if you were walking up to a person in a restaurant and asking them to speak softer on their phone. I really believe that there is a common ground where Google Glass can exist in public. It's called having faith in our fellow human beings. Innocent until proven guilty.
Society has adapted to technology just as quickly as technology has adapted to society over the years. I don't believe folks really tolerate the person screaming into their bluetooth earpiece whilst waiting in line for movie tickets. Looks, stares, glares and comments usually get the hint across. Callers have a responsibility of being respectful of those around them. Etiquette needs to be established. Technology shouldn't be banished. If people can't figure out etiquette then I believe we have a bigger problem in society today.
That all being said, Glass wears should be respectful of those around them. Wearing Google Glass in public in not unlike any other situation — betamax cameras at tourist traps, phones in theatres, etc. Societal pressure will lead where one can or cannot use Google Glass. Respect. It's not a difficult concept.
Remember that though something may be different and outside your comfort zone, it doesn't mean that it is wicked scary and must be stopped at once. Don't be afraid to think outside the box and maybe build a new one in the process.