XR Design & Development

Global Hackathon for Vision Pro Development, Vision Hack, Kicks Off Next Month

August 1, 2024 From roadtovr

Vision Hack is a forthcoming remote hackathon for developers building apps for visionOS. Open to all skill levels, the event is set to take place September 13–15th.

Guest Article by Cosmo Scharf

Cosmo Scharf is an Emmy-nominated product designer and entrepreneur with a decade of experience in XR. He co-founded Vision Hack and visionOS Dev Partners to support visionOS developers. Previously, Scharf created MovieBot, an AI-powered 3D animation app, and Mindshow, a VR platform for animated content creation. His also started VRLA, one of the world’s largest immersive technology expos.

Imagine building the first iPhone apps in 2008. That’s where we are with visionOS today. Vision Hack is your launchpad into the new frontier of spatial computing.

Over the past decade, I’ve had the privilege of witnessing the virtual reality industry evolve from a niche technology to a transformative medium. From the early days of clunky prototypes to the sleek, powerful devices we have today, the journey has been nothing short of extraordinary. Now, with the introduction of Apple Vision Pro, we’re standing at the threshold of a new era.

As one of the organizers of Vision Hack, I’m thrilled to announce the launch of the first global visionOS hackathon. Scheduled for September 13–15th, this event represents a significant milestone in our industry’s progression. It’s an opportunity to explore and shape the future of spatial computing as Apple Vision Pro continues its global rollout.

Vision Hack is designed to be a truly immersive experience. We’re encouraging teams to communicate in Vision Pro itself using spatial Personas, in addition to Discord. This approach not only showcases the device’s capabilities but also provides participants with authentic, hands-on experience in visionOS development.

Our three-day program caters to both seasoned spatial computing developers and newcomers:

Day 1: Workshops and team formation
Day 2: Intensive development with mentorship from industry experts
Day 3: Development, project presentations, and awards

To foster collaboration while ensuring focused development, we’ve capped team sizes at 5 people each. Understanding the global nature of our community, we’re organizing local meetups in various cities so developers can connect in person.

While we highly recommend access to a Vision Pro for the full experience, it’s not a strict requirement for participation. However, developers will need a Mac with an Apple chip to run the visionOS simulator. This setup will enable meaningful participation even without the physical device.

The organizing team brings extensive experience from major VR expos (VRLA), Metaverse hackathons, and XR startups.

As spatial computing evolves, we believe early developer engagement is crucial in building a robust ecosystem. Vision Hack aims to play a key role in nurturing the visionOS developer community, potentially influencing the trajectory of spatial computing applications.

For developers keen on exploring visionOS, Vision Hack offers a unique opportunity to dive into this emerging platform. There’s a $25 registration fee, which helps us cover some of the event costs and ensures committed participation.

For companies interested in being at the forefront of spatial computing development, we offer various sponsorship opportunities. These partnerships not only support the event but also provide sponsors with direct access to a pool of talented developers working on cutting-edge spatial computing applications.

More details, registration information, and sponsorship opportunities can be found at visionoshackathon.com. We’re excited to see the innovative projects and ideas that will emerge from this event, and we look forward to welcoming you to the next chapter of spatial computing development.

Quest ‘Augments’ Feature for Concurrent AR Apps Needs More Time to Cook, Says Meta CTO

June 18, 2024 From roadtovr

Last year Meta announced the so-called Augments feature, planned for Quest 3, which would allow persistent mini AR apps to live in the world around you. Now, eight months after the headset hit store shelves, Meta’s CTO explains why the feature has yet to ship.

Augments was announced as a framework for developers to build mini AR apps that could not just live persistently in the space around you, but also run concurrently alongside each other—similar to how most apps work on Vision Pro today.

And though Meta had shown a glimpse of Augments in action when it was announced last year, it seems the company’s vision (and desire to market that vision) got ahead of its execution.

This week Meta CTO Andrew “Boz” Bosworth responded to a question during an Instagram Q&A about when the Augments feature would ship. He indicated the feature as initially shown wasn’t meeting the company’s expectation.

We were playing with [Augments] in January and we decided it wasn’t good enough. It was too held back by some system architecture limitations we had; it ended up feeling more like a toy and it didn’t really have the power that we think it needed to deliver on the promise of what it was.

So we made a tough decision there to go back to the drawing board, and basically [it needed] a completely different technical architecture. Starting from scratch basically. Including actually a much deeper set of changes to the system to enable what we wanted to build there. I think we made the right call—we’re not going to ship something we’re not excited about.

But it did restart the clock, and so [Augments is] going to take longer than we had hoped to deliver. I think it’s worth while, I think it’s the right call. But that’s what happened.

We’re only two-and-a-half months out from Meta Connect 2024, which would be the one-year anniversary of the Augments announcement. That’s where we likely to hear more about the feature, but at this point it’s unclear if it could ship by then.

VisionOS 2 Enables WebXR by Default, Unlocking a Cross-platform Path to Vision Pro

June 12, 2024 From roadtovr

We’ve know that Apple planned to support WebXR for quite some time, but with VisionOS 2, the company is enabling the feature for all users. WebXR allows developers to deliver cross-platform XR experiences directly from the web, with no gatekeepers to approve or reject content.

WebXR is the widely supported web standard that allows developers to deliver AR and VR content from the web.

Just like anyone can host a webpage online without any explicit approval from another party, WebXR allows the same for AR and VR content. And because it’s delivered through the browser, accessing and sharing WebXR experiences is as easy as clicking or sending a link—like this one.

Vision Pro has supported initial WebXR support since its launch, but it required users to manually enable the feature by digging into Safari’s settings.

With VisionOS 2—available today as a developer preview, and coming to all this Fall—WebXR will be enabled by default, making it easy for anyone to access WebXR through the headset. Vision Pro thus joins headsets like Quest, HoloLens 2, and Magic Leap 2 in supporting WebXR content.

Though WebXR is “supported” on VisionOS 2, our understanding is that it only support VR (or ‘fully immersive’) experiences. WebXR is also capable of delivering AR experiences (where virtual content is merged with a view of the real world), but VisionOS 2 doesn’t yet support that portion of the standard.

There’s many reasons why developers might want to use WebXR to build experiences over native apps that are distributed through a headset’s proprietary store.

For one, any headset with WebXR support can run any compatible WebXR experience, meaning developers can build one experience that works across many headsets, rather than needing to make multiple builds for multiple headsets, then uploading and managing those builds across multiple platform stores.

Like a webpage, WebXR content can also be updated at any time, allowing developers to tweak and enhance the experience on the fly, without needing to upload new builds to multiple stores, or for users to download a new version.

WebXR also has no gatekeepers. So content that wouldn’t be allowed on, say, Apple or Meta’s app stores—either for technical or content-related reasons—can still reach users on those headsets. That could include adult content that’s explicitly forbidden on most platform app stores.

VisionOS 2 Improvement Targets Key Vision Pro Critique Among Developers

June 12, 2024 From roadtovr

For basic Vision Pro interactions like navigating apps and scrolling web pages, the headset’s look-and-pinch input system works like magic. But if you want to go more ‘hands-on’ with virtual content, the headset’s full hand-tracking leaves much to be desired.

Compared to Quest 3, Vision Pro’s full hand-tracking has notably more latency. That means when moving your hands it takes longer for the headset to register the movement. Especially in interactive content where you directly grab virtual objects, this can make the objects feel like they lag behind your hand.

Changes coming in VisionOS 2 stand to improve hand-tracking. Apple detailed the changes in a developer session at WWDC 2024 this week.

For one, the headset will now report estimated hand positions at 90Hz instead of the previous 30Hz. That means the system can reflect changes in hand position in one-third of the time, also making the movement of the hand smoother thanks to more frequent updates. This only applies to a small portion of the overall latency pipeline (which we previously estimated at a total of 127.7ms) but it could reduce hand-tracking latency by as much as 22ms in the best case scenario.

Here’s a look at that in action:

It’s an improvement, but you can still easily see the latency of the teapod compared to the hand, even with this slow movement.

For a snappier experience, VisionOS 2 will alternatively allow developers to enable hand-tracking prediction, which provides an estimate of the user’s future hand position. While this doesn’t truly reduce latency, it can reduce perceived latency in many cases. Similar prediction techniques are common across various XR tracking systems; it’s quite surprising that Vision Pro wasn’t already employing it—or at least not making it available to developers.

Here’s a look at predictions in action:

Now we can see the virtual teapot staying much more aligned to the user’s hand. Granted, this isn’t likely to look quite as good with faster motions.

We’ll be looking forward to putting Vision Pro’s hand-tracking latency to the test with VisionOS 2 soon!

Some of Vision Pro’s Biggest New Development Features Are Restricted to Enterprise

June 11, 2024 From roadtovr

VisionOS 2 is bringing a range of new development features, but some of the most significant are restricted to enterprise applications.

VisionOS 2 will bring some of the top requested development features to the headset, but Apple says its reserving some of them for enterprise applications only.

Developers that want to use the features will need ‘Enterprise’ status, which means having at least 100 employees and being accepted into the Apple Developer Enterprise Program ($300 per year).

Apple says the restriction on the new dev capabilities is to protect privacy and ensure a predictable experience for everyday users.

Here’s a breakdown of the enterprise-only development features coming to VisionOS 2, which Apple detailed in a WWDC session.

Vision Pro Camera Access

Up to this point, developers building apps for Vision Pro and VisionOS couldn’t actually ‘see’ the user’s environment through the headset’s cameras. That limits the ability for developers to create Vision Pro apps that directly detect and interact with the world around the user.

With approval from Apple, developers building Vision Pro enterprise apps can now access the headset’s camera feed. This can be used to detect things in the scene, or to stream the view for use elsewhere. This is popular for ‘see what I see’ use-cases, where a remote person can see the video feed of someone at a work site in order to give them help or instruction.

Developers could also use the headset’s camera feed with a computer vision algorithm to detect things in view. This might be used to automatically identify a part, or verify that something was repaired correctly.

Even with Apple’s blessing to use the feature, enterprise apps will need to explicitly ask the user for camera access each time it is used.

Barcode and QR Code Detection

Being able to use the headset’s camera feed naturally opens the door for reading QR codes and barcodes, which allow structured data to be transmitted to the headset visually.

Apple is providing a readymade system for developers to detect, track, and read barcodes using Vision Pro.

The company says this could be useful for workers to retrieve an item in a warehouse and immediately know they’ve found the right thing by looking at a barcode on the box. Or to scan a barcode to easily pull up instructions for assembling something.

Neural Engine Access

Enterprise developers will have the option to tap into Vision Pro’s neural processor to accelerate machine learning tasks. Previously developers could only access the compute resources of the headset’s CPU and GPU.

Object Tracking

Although the new Object Tracking feature is coming to VisionOS 2 more broadly, there are additional enhancements to this feature that will only be available to enterprise developers.

Object Tracking allows apps to include reference models of real-world objects (for instance, a model of a can of soda), which can be detected and tracked once they’re in view of the headset.

Enterprise developers will have greater control over this feature, including the ability to tweak the max number of tracked objects, deciding to track only static or dynamic objects, and changing the object detection rate.

Greater Control Over Vision Pro Performance

Enterprise developers working with VisionOS 2 will have more control over the headset’s performance.

Apple explains that, out of the box, Vision Pro is designed to strike a balance between battery life, performance, and fan noise.

But some specific use-cases might need a different balance of those factors.

Enterprise developers will have the option to increase performance by sacrificing battery life and fan noise. Or perhaps stretch battery life by reducing performance, if that’s best for the given use-case.

There’s more new developer features coming to Vision Pro in VisionOS 2, but these above will be restricted to enterprise developers only.

VisionOS 2 is Available in Developer Preview Starting Today

June 10, 2024 From roadtovr

Today at WWDC 2024, Apple revealed VisionOS 2, the first major update for its Vision Pro headset. The new version of the software will be available for developers to experiment with starting today.

VisionOS 2 is primarily designed to round out some rough edges from the headset’s release earlier this year, while adding some new features and also expanding development capabilities so developers can take greater advantage of the headset.

While it won’t release publicly until the fall, Apple says developers can get their hands on VisionOS 2 starting today. We haven’t spotted the direct page to the developer preview update yet, but the company says it will be available through its official developer website.

VisionOS 2 is bringing a range of new features and improvements like 2D to 3D photo conversion, ultrawide Mac Virtual Display, new ways to navigate the headset’s core interface, and much more. We’ll be breaking down the full range of features soon, stay tuned to the front page!

OpenXR 1.1 Update Shows Industry Consensus on Key Technical Features

April 17, 2024 From roadtovr

OpenXR, the open standard that creates a standardized way for XR hardware and applications to interface, has seen its first major update. OpenXR 1.1 evolves the standard by incorporating new functionality that was important to the industry but previously not standardized.

Facilitated by the standards body Khronos Group, OpenXR is a royalty-free standard that aims to standardize the development of VR and AR applications, making for a more interoperable ecosystem. The standard has been in development since April 2017 and over time has become supported by virtually every major hardware, platform, and engine company in the VR industry, including key AR players—but notably, not Apple.

Following the OpenXR 1.0 release in 2019, this week’s release of OpenXR 1.1 is the first major update to the standard in more than four and a half years.

The update shows the standard evolving as industry needs emerge, an outcome that’s part of the standard’s design.

Built into the framework of OpenXR is the notion of ‘extensions’, which are vendor-specific capabilities which can customize OpenXR’s functionality without needing to first go through the process of being baked into the official standard.

In some cases, such extensions include functionality that eventually becomes universal enough to warrant inclusion in the standard overall. Thus, extensions can be ‘promoted’ and baked into the OpenXR standard for all to use and support.

OpenXR 1.1 sees the inclusion of five capabilities which originally started as extensions:

Local Floor: provides a new Reference Space with a gravity-aligned world-locked origin for standing-scale content that can be recentered to the current user position at the press of a button without a calibration procedure. It also has an estimated floor height built-in. More details about Local Floor functionality and its value to developers are available in this blog post.

Stereo with Foveated Rendering: provides a Primary View Configuration to realize eye-tracked foveated rendering or fixed foveated rendering for XR headsets across multiple graphics rendering APIs. Its use is especially beneficial for efficiently rendering high-pixel-count displays, which put a heavy load on the GPU. The original vendor extension has been adopted natively in Unity, Unreal, and recently by NVIDIA Omniverse.

Grip Surface: provides a Standard Pose Identifier that reliably anchors visual content relative to the user’s physical hand, whether the hand position is tracked directly or inferred from a physical controller’s position and orientation.

XrUuid: provides a Common Data Type to hold a Universally Unique Identifier that follows the IETF RFC 4122.

xrLocateSpaces: provides a Locating Spaces function to improve performance and simplify application code by enabling an application to locate an array of spaces in a single function call populating an “array of structures” (AoS), instead of being limited to locating a single space per function call.

Building these extensions directly into OpenXR represents the industry’s consensus on the demand for these features and how they should be implemented across the ecosystem.

OpenXR 1.1 also includes various improvements to existing features and clarifies some capabilities to make the standard clearer for those that want to build implementations that conform to the standard.

Going forward, the OpenXR working group (consisting of representatives from member companies which steer the standard) says it plans to make more regular updates to OpenXR going forward, ensuring that new capabilities continue to be added as industry needs evolve.

“OpenXR 1.1 marks a significant milestone in the development of this open standard that has become widely adopted throughout the XR industry. OpenXR 1.0 provided baseline capabilities and the foundation for experimentation with new functionality through extensions,” says Alfredo Muniz, Chair of the OpenXR Working Group. “Now the Working Group is pivoting to manage regular core specification updates that balance the need for flexibility to ship new functionality with consolidation of proven technology to reduce fragmentation and enable true cross-platform application portability.”

Vision Pro and Quest 3 Hand-tracking Latency Compared

March 28, 2024 From roadtovr

Vision Pro is built entirely around hand-tracking while Quest 3 uses controllers first and foremost, but also supports hand-tracking as an alternate option for some content. But which has better hand-tracking? You might be surprised at the answer.

Vision Pro Hand-tracking Latency

With no support for motion controllers, Vision Pro’s only motion-based input is hand-tracking. The core input system combines hands with eyes to control the entire interface.

Prior to the launch of the headset we spotted some footage that allowed us to gauge the hand-tracking latency between 100-200ms, but that’s a pretty big window. Now we’ve run our own test and more precisely find Vision Pro’s hand-tracking to be about 128ms on visionOS beta v1.1.1.

Here’s how we measured it. Using a screen capture from the headset which views both the passthrough hand and the virtual hand, we can see how many frames it takes between when the passthrough hand moves and when the virtual hand moves. We used Apple’s Persona system for hand rendering to eliminate any additional latency which could be introduced by Unity.

After sampling a handful of tests (pun intended), we found this to be about 3.5 frames. At the capture rate of 30 FPS, that’s 116.7ms. Then we add to that Vision Pro’s known passthrough latency of about 11ms, for the final result of 127.7ms of photon to hand-tracking latency.

We also tested how long between a passthrough tap and a virtual input (to see if full skeletal hand-tracking is slower than simple tap detection), but we didn’t find any significant difference in latency. We also tested in different lighting conditions and found no significant difference.

Quest 3 Hand-tracking Latency

How does that compare to Quest 3, a headset which isn’t solely controlled by the hands? Using a similar test, we found Quest 3’s hand-tracking latency to be around 70ms on Quest OS v63. That’s a substantial improvement over Vision Pro, but actual usage of the headset would make one think Quest 3 has even lower hand-tracking latency. But it turns out some of the perceived latency is masked.

Here’s how we found out. Using a 240Hz through-the-lens capture, we did the same kind of motion test as we did with Vision Pro to find out how long between the motion of the passthrough hand and the virtual hand. That came out to 31.3ms. Combined with Quest 3’s known passthrough latency of about 39ms that makes Quest 3’s photon to hand-tracking latency about 70.3ms.

When using Quest 3, hand-tracking feels even snappier than that result suggests, so what gives?

Because Quest 3’s passthrough latency is about three-and-a-half times that of Vision Pro (11ms vs. 39ms), the time between seeing your hand move and your virtual hand move appears to be just 31.3ms (compared to 116.7ms on Vision Pro).

– – — – –

An important point here: latency and accuracy of hand-tracking are two different things. In many cases, they may even have an inverse relationship. If you optimize your hand-tracking algorithm for speed, you may give up some accuracy. And if you optimize it for accuracy, you may give up some speed. As of now we don’t have a good measure of hand-tracking accuracy for either headset, outside of a gut feeling.

Apple is Adding Support for Vision Pro’s Input System to WebXR

March 20, 2024 From roadtovr

Apple is adding support for Vision Pro’s unique input system to WebXR, the web standard which allows XR experiences to run right from a web browser.

One of the most unique things about Apple Vision Pro is its input system which eschews motion controllers in favor of a ‘look and pinch’ system which combines eye-tracking with a pinch gesture. On the whole it’s a really useful way to navigate the headset, but because it works so differently than motion controllers, it doesn’t play too well with WebXR.

But Apple is working to fix that. This week the company announced the latest version of VisionOS (1.1) includes a new input mode for Safari’s WebXR capabilities called ‘transient-pointer’. This new mode provides inputs from the headset in a standardized way which developers can use to understand what users are selecting inside of a WebXR session running on Vision Pro.

Up to this point, WebXR apps typically expect a headset report a continuously updated position of each controller. But Apple says it built Vision Pro’s input system to reveal as little information about the user as possible, so it doesn’t report the pose or position of the user’s hands by default. Instead, it only reveals such information at the moment of the user’s pinch (though it’s possible for a WebXR app to ask for full hand tracking info).

With the new transient-pointer option, when a user pinches the WebXR app will be able to see a ray representing the direction of the user’s gaze and the coordinate position of their pinch. Like in VisionOS itself, the app thus looks at the pinch to decide ‘when’ a user is making an input, and looks at the ray to decide ‘where’ they’re making the input.

For the duration of the pinch, the position of the pinch itself is continuously updated, allowing for interactions like dragging, pushing, and pulling objects. But when the pinch is released, the app no longer has access to the direction the user is looking or where their hand is located.

With these new capabilities, WebXR apps will be able to adapt their interactions to work correctly with Vision Pro.

However, WebXR on Vision Pro is still experimental. Developers must manually enable WebXR capabilities by accessing advanced settings of Safari in the headset. Developers can also experiment with WebXR and the transient-pointer mode using the VisionOS simulator.

The transient-pointer mode for Vision Pro is being baked into the WebXR standard, and has been added to the most recent draft version of the specification. That means that devices which adopt the same input mode will be able to tap into the same WebXR capabilities.

This is What a Vision Pro Competitor From Meta Could Look Like

February 5, 2024 From roadtovr

With Apple Vision Pro out in the wild, it’s likely that Meta’s Quest Pro line will evolve to become more of a direct competitor. And we’ve actually got an idea of what an even higher-end Quest Pro 2 could potentially look like.

Apple’s Bar

With Vision Pro, Apple was clearly not aiming to make an affordable headset. Their strategy was to create the minimum viable quality bar for what they wanted the mixed reality experience to look and feel like. After packing in all the tech & features they thought was necessary, Vision Pro landed at $3,500.

That makes it only an indirect competitor at best to something like the $500 Quest 3. But clearly there’s a market of people willing to drop substantial money for a high-end mixed reality headset.

The Future of Quest Pro

So it makes sense that Meta’s Quest Pro line will evolve to compete in that high-end price range. But what could a Quest Pro 2 look like if Meta wanted to go toe-to-toe with Apple?

Well, it turns out we might have a pretty good idea.

Douglas Lanman is the Senior Director of Display Systems at Reality Labs Research, the R&D arm of Meta’s XR division. Late last year he gave a guest lecture at the University of Arizona, a school from which Reality Labs regularly recruits researchers.

Lanman spent most of the one-and-a-half hour session overviewing the research his team has worked on in recent years. They’ve built various prototypes separately exploring holographic optics, high-dynamic range, retina resolution, reverse-passthrough, and more.

While most of the headsets presented were purely research prototypes, Lanman wrapped up the session by talking about his vision for ‘Mirror Lake’, a concept headset that could incorporate many of the technologies his team has been exploring over the last decade. He presented the following rendering of what the headset could look like if it was actually built today.

Video via Mixed Reality News

And while he stressed this was just a concept dreamt up by him and his team—not a product roadmap—he said the concept was “practical to build now” with parts that could be readily sourced from the market. Whether or not it could be mass produced in the ballpark of $3,500 is an unanswered question, but Lanman said he thinks the industry is ready to reach “a new plateau” with a device like Mirror Lake.

He went on to explain that Mirror Lake would include compact holographic optics, multi-view eye-tracking, a varifocal display, reverse-passthrough, and baked-in prescription correction. That’s quite a mouthful… so let’s break each one of these down.

Holographic Optics

A huge challenge in making XR headsets compact is the need for optics to be placed between the viewer and the display. While Meta’s latest Quest Pro and Quest 3 headsets have impressively compact ‘pancake’ optics, Meta researchers think they can go even thinner. Their solution is a ‘holographic lens’, which is exactly what it sounds like: a lens baked into a hologram that can be practically as thin as a sheet of paper.

Multi-view Eye-tracking

Eye-tracking works best when you have multiple views of each eye. This means better source data for determining exactly which direction the eye is facing. But more cameras means more overhead for processing, heat, power, and cost. So Meta researchers dreamt up an interesting method of getting multiple eye-view of an eye from just one camera. By using a holographic element, the camera can look at the lens of the headset and see multiple reflections of the wearer’s eye. That could provide more views for better eye-tracking without adding more cameras.

Varifocal Display

Every XR headset on the market today uses stereoscopy—the overlapping of two similar images—to display 3D imagery. And while this is similar to how we see in the real world, it leaves out an important visual cue; the distance from which light originates changes how our eyes focus on what we’re trying to see. But because XR headsets have a fixed display, the light is always coming from the same distance. That means you can only set one focal distance, which would be like your eyes only being able to focus at once specific distance from you. This is known as the vergence-accommodation conflict (or VAC).

A varifocal display is any display which allows variable focus, thus solving for VAC. Meta has researched several solutions to this problem, but the one Lanman says could be part of Mirror Lake is using a stack of electronically controlled polarizers to allow dynamic alteration of the lens focus. Combined with eye-tracking, this would allow the system to focus the light for the specific part of the scene that you’re looking at.

Reverse-passthrough

‘Passthrough’ is what we call it when you put cameras on the outside of a headset and use them to show the wearer a view of the outside world. It’s kind of like your vision is ‘passing through’ the headset. Reverse-passthrough is what we call it when you put cameras inside the headset to show the outside world a view of your face. This is the same thing you may have seen on Vision Pro (Apple calls it ‘EyeSight’).

But it isn’t as easy as just sticking a display to the outside of the headset. Because the front of your headset sits quite far from your actual eyes, showing an image of your eyes that far out would look very unrealistic—like your eyes were somehow glued to the front of the headset.

Instead, you need a way to make the eye appear sunken back into the headset. To do that you need some kind of light-field display, which is a display that shows different views depending upon the angle that you’re looking at.

Lanman said that in the Mirror Lake render they actually did a ray-tracing simulation to show an approximation of what the reverse-passthrough on the headset would look like, given the components the headset would theoretically be made from.

Prescription Baked-in

For those who need glasses, having a headset that can support your prescription is important. Lanman says that given the compact nature of Mirror Lake, there isn’t much room for glasses. Instead he suggests that such a headset would be customized for each user’s specific eyesight needs. On Mirror Lake that could be achieved by manufacturing the holographic lenses to accommodate vision correction for each individual. That’s clever because it means correcting the headset’s visuals for each user without adding any additional bulk or components to the headset.

– – — – –

Meta’s first ‘Pro’ headset, Quest Pro, had an awkward launch to say the least. While it had some impressive optics, face-tracking, and improved mixed reality capabilities, it still end up feeling like a souped-up Quest 2. It played all the same content and the few things it did uniquely didn’t quite feel like they justified the $1,500 price point. Meta seemed to agree, because it dropped the headset to $1,000 not long after release.

Making Quest Pro’s launch even more rocky, Quest 3 was announced not long after and brought many of Quest Pro’s improvements down to a much more attractive $500 price point.

If Meta is going to have a ‘Pro’ headset line, the headset needs to do substantially more than its consumer-focused headsets. Vision Pro—packed with tech and priced at $3,500—has seemingly opened the door to Meta further differentiating future Quest Pro headsets with a higher-end offering.

And considering that one of Meta’s senior researchers think it’s possible to build a Mirror Lake-like headset with parts that are already available, it’s not a leap to think that some of that new tech could find its way into a Quest Pro 2 or Quest Pro 3.