John Carmack, a brilliant, gifted developer, recently said this about V/AR, and it continues to bother me:

I remain unconvinced that mixed reality applications are any kind of an engine for increasing headset sales. High quality pass through is great, but I just don’t see applications built around integrating rendering with your real world environment as any kind of a killer app. I consider it interesting and challenging technology looking for a justification.

But we already have mixed reality applications. AirPods in transparency mode are one, but even transitor radios at baseball games in the late 50s. Telephones? They don't render sight, but they do render augmented soundscapes. Ask the blind if a sightless world counts as a reality.

And what are our phones? They aren't VR. They do interact. Take the blue dot on a map -- or the location-based notifications we get walking into an Apple store.

Do I just not understand what Mixed Reality is? Apparently not precisely.

Tom's hardware tries to explain:

Virtual reality (VR) is a fully immersive world that is created by hardware that does not bring in elements of the real world.

...

Augmented reality, meanwhile, is the other end of the spectrum — which Microsoft refers to as the mixed reality spectrum. Augmented reality (AR) basically adds digital overlays to the existing real world.

...

Mixed reality (MR) lies somewhere in between the two. It adds overlays and real-world objects into a virtually rendered world. Intel actually describes it really well. “You can play a virtual video game, grab your real-world water bottle, and smack an imaginary character from the game with the bottle.” In proper mixed reality, the lines completely blur.

If you say so. I mean, our location is already "mixed" into the virtual world of Pokemon Go. And as much as you try, if the imaginary character tries to let you drink out of their imaginary water bottle, you'll stay thirsty, my friends.

MR seems to be another way of saying "AR, but, like, really augmented" -- as in there's a blurring of the lines between virtual and real, not simply a small or passive overlay. That is, to me, it sounds a little like marketing for AR. Again, the "mixed" water bottle will not satisfy "real" thirst. It remains an augmentation onto, not a physical replacement.

Let's say a rational take is this: VR is as dissociated from the "real"/traditional/legacy world experience as possible. Your actual surroundings are [idealistically] of zero value. In a sense, you're in a perfect, valueless heterotopion so that you can create and experience something completely ungrounded.

AR foregrounds the now, the legacy, and adds to it. Again, think transistor radio while you're at the ballpark so Vin Scully can tell you what you need to know about the game you're watching on the field. But also the silly HUD dossiers' that appear when someone pops into view in spy or Terminator movies (apparently called Termovision!).

MR seems to have lost any meaning. It's not VR, so it's a [marketing?] spin on AR. Maybe it has a very real C-3PO in your visual field and you can speak to them like it's real or they move out of the way of "real" oncoming cars. I don't know. I can't think of a phenomenon where there's a meaningful space between "not VR" (where VR equals "completely dissociated from the real") and "AR" (where something is overlayed on/augementing the "real") for mixed reality to live. MR is AR. Any "R" that's not "VR" is "AR".

So I'm back to wondering what Carmack is talking about when he's down on "mixed reality".

Maybe it's the immersion. There, I'd tell him to lower his bar. You don't have to have Apple Vision to have mixed reality. Again, any interface that puts the digital (okay, okay, "sensory"; doesn't have to be digital) into the "real" works. The McDonald's app that lets you order food. Your EZ Pass transponder.

Does Mixed Reality require glassses? Then why doesn't it require earphones? How about gloves and sleeves and suits that remediate touch? Is his argument that AR glasses aren't MR enough and are, therefore, doomed to fail?

If so, the critique reduces to the absurd: "There is no MR killer app until the digital IS REAL!!1!"

I think his metric for MR success is literally what he says -- "headset sales". He'd say, sure, MR sells phones, radios, EZ Pass accounts, but not glasses. It'll never sell glasses.

I'm wearing glasses typing on a laptop right now. If I could get my phone and computer into that same shape without a concern for weight or power (or eyesight), I would. The potential isn't just there, it's at least as big as the market for glasses.

I don't know about Carmack, but I'm glad Apple has started looking for a way to make those glasses happen.

Labels: apple, apple vision, AR, business, johnc, VR