VR prototypes reveal Facebook’s surprisingly critical research directions

Not long ago, Tested posted a video about practical time with prototypes for virtual reality (VR) headsets from Meta (that is Facebook), and there are some genuinely interesting pieces in there. The video itself is over an hour long, but if you are primarily interested in the technical angles and why they mean something to VR, read on because we want to highlight each of the main points of the research.

As absurd as it may seem to many of us to have a social network at the forefront of meaningful VR development, one cannot say that they do not take it seriously. It is also refreshing to see each of the prototypes displayed by a researcher who is clearly excited to talk about their work. The big dream is to find out what it takes to pass the “visual Turing test”, which means delivering images that are on a par with a physical reality. Some of these critical elements may come as a surprise, because they go in directions beyond resolution and field of view.

Solid-state varifocal lens demo, capable of 32 discrete focal steps.

At 9:35 on the video, [Douglas Lanman] shows [Norman Chan] how important variable focus is to deliver a good visual experience, followed by a review of all the different prototypes they have used to make it happen. Currently, VR headsets display visual images on only one focal plane, but this means that – among other things – bringing a virtual object close to one’s eyes becomes blurred. (By the way, older people do not think that part is very strange because it is a common side effect of aging.)

The solution is to change the focus based on where the user sees, and [Douglas] shows all the different ways this has been explored: from motors and actuators that mechanically change the focal length of the screen, to a solid-state solution composed of stacked elements that can selectively converge or diverge light based on polarization. [Doug]the pride and excitement is palpable, and he goes into great detail on everything.

At 30:21, [Yang Zhao] explains the importance of higher resolution screens, and also talks about lenses and optics. Interestingly, the ultra-clear text reproduction made possible by a high-resolution screen is what ended up capturing [Norman]his attention most. When high resolution was combined with variable focus, it was the textures on the pillows, the liveliness of the wall art and the patterns on the walls that [Norman] found that he just could not stop exploring.

Next up at 39:40 is something very interesting, shown off [Phillip Guan]. A VR headset must use software corrections for distortions, and it turns out that these corrections can be complex. Not only does an image get a certain amount of distortion when it passes through a lens, but that distortion Changes in nature depending on where the eye sees. All of this needs to be corrected for software for a high-fidelity experience, but a real bottleneck is waiting for a physical prototype to be constructed, and complicating this is that different people will have slightly different subjective experiences of distortion. To solve this, [Phillip] shows a device whose purpose is to accurately simulate different physical headset designs (including different lenses and users) in software, allowing exploration of different designs without actually building anything.

The final prototype – called Starburst for reasons that will soon be ready – is shown at 44:30 and demonstrates the power of real high dynamic range. It is the most unwieldy look, but it is mainly due to having headlights as a taillight. The purpose is not to blind users, but to deliver something important and missing. Why is high brightness so important? The answer is simple: real-world light levels are far above anything a modern monitor (or VR headset) can deliver. This means that in VR, a spotlight only looks like one picture of a spotlight. It will never really look like that light, not in the way your eyes and brain actually experience the word. When headsets can deliver a true HDR experience, that will change, and that’s what this prototype delivers.

It is clear that this direction is taken very seriously, and it may come as a surprise to hear that delivering a compelling visual experience goes significantly beyond higher resolution and wider field of view. All the really good VR ideas may have been dreamed up back in the 1960s, but this video is a great showcase for what goes into the tough scientific work of figuring out how to solve a problem.