Most social activity will reside in VR by 2036

By mako yass @ 2022-08-22T19:25 (+16)

(Gravestone: The title used to read "- by 2030". It was a bad title. I'm surprised to find that I might actually believe the 2030 target, but the factors that get us there are weird and cultural, and I've mostly only presented the much more legible technological factors here, so never mind 2030 for now. I'll try to examine that case properly later.)

This post was written as a resource for a set of project proposals in VR communitybuilding and transformative social technology design, but it should be of interest to anyone who'd like to have a rough idea of how our world will start to change very soon.

I should first make it clear what kind of devices we're talking about: The VR headsets of 2025 (or potentially even next year) will be qualitatively different from what we have today:

• Apple and Meta are known to be releasing headsets in 2023 with
  • face tracking, the ability to capture and convey the full facial expressions of the user, including,
  • gaze tracking (which may potentially obsolete the use of mice in most kinds of work^[1])
  • "passhthrough", the use of external cameras to allow the user to see and interact with the external world while they're operating in the virtual world. (Meta's Quest 2 already partially does). At some point, there wont be much of a functional distinction between virtual reality and augmented reality headsets.
  • Potentially, varifocal lenses, which solve an eye strain issue, and potentially, if we're lucky, will avert any role VR might have played as a cause of myopia.
• VR displays are on a trajectory to hit the maximum perceptible level of visual detail (60 pixels per degree) by 2026. There are plausible cases that next year's apple headset will already be there^[2] (although Apple's headset wont be immediately transformative, as it seems like they wont be able to build enough to meet demand immediately, unless the plan has changed recently).
• They're getting lightweight. "Pancake lenses" will reduce the weight and volume of optics. Unlike today's high-end headsets, it will be easy to forget that you're wearing anything at all, comfort will be less of an obstacle.

I think that adds up to a generally preferable modality for computing, functionally equivalent to having an extremely large computer screen that also weighs nothing, fits anywhere, and moves with you. The prospect of continuing this practice of coupling a relatively small screen to your keyboard then hunching over it all day (I'm describing laptops) seems like it will be difficult to continue to justify. Desktops on standing desks hold up a bit better, but...

There are crucial practical activities that just aren't possible without VR:

• Many kinds of exercise games. Some people don't have access to nice places to go running or cycling, or they aren't able to find joy in it. I have found that exercise games make it much easier to push myself far beyond the point I'd normally go by projecting meaning onto the actions, EG: fleeing, chasing, fighting, or cutting virtual wood.
• A much better modality for any work that requires manipulating 3d objects in space. CAD work, for instance (which I'd expect to make up an increasing proportion of the GDP as additive manufacturing improves, as manufacturing automation increases, or just, as production processes generally become more accessible to more people.)^[3]
• Any sort of remote work: Having a sense of presence in a focused office (or at least, a sense of being in a busier place than the home) is important enough for getting focused work done, that people will sometimes pay for seats at coworking spaces even when they're working alone. VR can grant you the focus of being immersed in an office without a commute, without the fee, and the office will be fantastically better than any real office, it can be linked to other offices in non-euclidean ways, it can be restructured and rearranged cheaply, it can look and behave in impossibly magical ways to support impossibly magical vibes, and also you can't get respiratory diseases in a virtual office.
• Connecting with others remotely in general is a much better experience in VR than it is over Zoom! Reasons discussed more below. This is where most of the social transformation is going to come from.

So I think basically everyone will end up having or wanting or needing one or being hounded to get one by their friends (I have never seen so many people so willing to buy their friends multi hundred dollar devices), once we all start to notice that VR is really strongly preferable over text chats or video chats, for meeting new people, hanging out, or talking things through. I'm surprised and not totally sure why VR is already so preferable for me over video chats, but it probably has a lot to do with the following technical advantages:

• Spacial, directional audio, being able to implicitly convey your focus of attention by facing, and walking around the space, allows group conversations to organically split and merge. This is extremely important for anything aspiring to work in the way of a meetup, party or conference.
• Some body language. Pointing, gesticulation, moving around. (Full body language should arrive with camera-based tracking, and can be had today with special equipment.)
• Non-disadvantage: Although face-tracking will start to deploy next year, lacking face tracking isn't as big of an issue as you might expect, as the information that facial expressions convey is mostly already transmitted in speech. Don't believe me? Check out Meta's surprisingly good expression-recovery-from-sound thing. I admit that it's not so effective that I'd actually want to use it, but it's an illustration of how much voice conveys.
• (Believe it or not, video chats could eventually catch up with VR on these features, with promotion of the use of headphones, specialized compression algorithms, or filters, but it hasn't yet, so I'll still mention them.)
  • Most people don't realize this (and that's a large part of the problem), but when you have an audio call without audio isolation (without wearing headphones), the system silences you when other people are talking in order to prevent audio feedback. This disrupts natural turntaking negotiation and makes very lively conversation very difficult (impossible?). VR headsets don't have this problem.
  • It's basically impossible to make eye contact in video chats. It's basically possible in VR (Eye tracking is coming, but head tracking is kind of close already enough). This is subtle but it's probably pretty important for establishing a sense of connection!
  • Not having to stream a load of video in and out (streaming relatively compact encodings of gesture or facial expression instead) often makes the data connection of VR chats more reliable.

In sum, VR gets us much more natural online conversations than prior mediaforms.

Textual online communities are good for their own reasons and wont go away, but humans are not natives to that format, in text, we have difficulty judging tone, processes don't come naturally, and it definitely makes it difficult to make friends: You can't see anyone, on some level you find it difficult to relate to them as real people. VR lets you to feel the reality of distant people. It's much easier after sharing space in VR to feel that you've met the person, and that they're a part of your world.

We highly recommend this People Make Games documentary about VR Chat if you want to see the extent of the vibrancy of VR as a social environment. I hope that near-future VR culture will be less awkward than this, hopefully the semiotics will be more self-aware and evolved and refined but personally I'm already vibing.

And we should expect VR offices to be productive: Immersion, in online social environments, is not a minor thing. The sense of being transported to another place, being bodily present with others, is important for committing with your whole self to the people you're with and the thing you're doing together. It's important for connecting with people, it's important for working with them: It's much much easier to focus on getting stuff done when you feel bodily present with people who're also doing it, who visibly expect you to do it too, and who are perceiving whether you are doing it.

So, yeah, I think it's gonna happen.

And, I can't write this up fully yet, but, there's stuff we should be doing about this. The things that are built in this new place, the standards we adopt, are going to be highly consequential. Social technologies determine the shape and the character of the communities that form under them, how they will interact (EG: How well AI research organizations will coordinate!) and how they will spend their time, and that basically determines everything.

We've been gathering technical principles, components and designs for a Social VR Engine that we'd expect to compete well against other projects and improve online social health. I kind of expect us to seek funding pretty soon. I will probably write publicly about those designs, as anyone attentive enough and apprehending enough to copy and deploy these specific world-improving social technologies would sort of necessarily be an ally as a result of being that way and doing that for us (but most likely no one will).

If I've gotten you curious about VR, if you decide to check it out, I strongly recommend visiting us at the EA VR group.

1. ^{^}

   There's some ambiguity as to whether eye tracking will obsolete mice in general. Eye tracking seems to have an insurmountable limit on its accuracy, the eye is always bouncing around somewhat, seemingly uncontrollably and (currently) unconsciously. I assume it has to. It may be possible to learn somewhat better control through practice, but I've been unable to find research on that. I expect some limit to hold.

   However, head tracking may be able to make up the difference, when it counts: https://precisiongazemouse.org/ I haven't tried this, it's not clear from the video demo whether it's preferable to a mouse. It seems to be using an eye tracker that is below optimal accuracy. I don't know whether the upcoming headsets are going to be near optimal, but I'd expect them to be closer. It's definitely going to be possible to accurately point and click on sufficiently large things without lifting our hands from our keyboards, which is pretty cool.

2. ^{^}

   Source: A report tells of Sony providing a pair of small 8K OLED screens for Apple, which is more than enough detail to satisfy even the sharpest human eyes.

   An aside: The report also tells of a mysterious, third, lower detail AMOLED screen. I seem to be the only one who thinks this, but I'm pretty sure the third screen is going to go on the front of the device, so that the headset can display a (potentially stylized) rendering of the wearer's eyes. Being unable to see the wearer's eyes when they're using VR is actually a pretty strong social obstacle to normalizing it! So we should expect Apple to care a lot about that! Note that the front of the device seems to be black (like a phone that is turned off). It absolutely could be a screen. The only alternative theory I've heard: that the third screen is going to be put behind the others and used for peripheral vision, seems really weird to me from a manufacturing perspective. I don't know how they'd make that work.

3. ^{^}

   Why hasn't VR penetrated CAD already? Because adequate VR headsets could not be bought for any price 5 years ago (every headset was either too low res or too heavy to want to use all day, iirc), and because CAD software is sophisticated and will take a while to adapt! Why didn't the industry prepare in advance of the arrival of today's headsets? Because tech forecasting skills are both rare and difficult to identify. But it is beginning to happen! Consider McLaren Vector Suite for instance.