ar industry

Qualcomm Reveals Snapdragon AR2 Processor for Glasses-sized AR Devices

November 16, 2022 From roadtovr

Qualcomm today announced Snapdragon AR2, its “purpose-built headworn augmented reality platform.” Differentiating from the company’s existing Snapdragon XR2 chips, Qualcomm says the AR2 architecture is better suited for creating AR glasses with low power consumption and compact form factors.

Today during Qualcomm’s Snapdragon Summit event, the company revealed the Snapdragon AR2 platform which consists of a trio of chips which the company says will help make truly glasses-sized AR devices possible.

Qualcomm was early to the standalone VR space and has been dominant with its Snapdragon XR2 chips which have found their way into many of the leading standalone headsets on the market, and are now in more than 60 devices total, the company says.

Aiming to take a similar bite out of the forthcoming AR glasses segment, Qualcomm has created a new Snapdragon AR2 platform with a distributed processing design. The platform consists of three chips:

AR processor (for sensor perception and video output)
AR co-processor (for sensor fusion and dedicated computer vision tasks)
Wi-Fi 7 chip (for communication to a host processing device)

By creating a more distributed workload across a main processor and a co-processor, Qualcomm claims AR2 is up to 50% more power efficient while offering 2.5 times better AI performance, and a more compact form-factor, compared to the single-chip Snapdragon XR2 solution.

Not only will the AR processor and co-processor help share a workload, Qualcomm also sees AR2 devices using the speedy Wi-Fi 7 chip to communicate with a host device like a smartphone or wireless compute puck that will do the heavy lifting like application processing and rendering. Qualcomm claims the Wi-Fi 7 chip (FastConnect 7800) can achieve 5.8 Gbps bandwidth with just 2ms of latency.

Using this three-chip framework for distributed processing, Qualcomm claims it will be possible to build compact AR glasses that consume less than one watt of power.

The AR2 platform supports up to nine concurrent cameras for a bevy of head-tracking, environment-sensing, and user-tracking tasks.

“We built Snapdragon AR2 to address the unique challenges of headworn AR and provide industry leading processing, AI and connectivity that can fit inside a stylish form factor,” said Hugo Swart, vice president of XR product management at Qualcomm. “With the technical and physical requirements for VR/MR and AR diverging, Snapdragon AR2 represents another metaverse-defining platform in our XR portfolio to help our OEM partners revolutionize AR glasses.”

There’s no word yet on when the first AR2 devices will hit the market, but Qualcomm lists a handful of partners actively working with the platform: Lenovo, LG, Niantic, Nreal, Oppo, Pico, Qonoq, Rokid, Sharp, TCL, Vuzix, and Xiaomi.

Varjo Raises $40M Series D Funding to Expand Cloud-based XR Platform

September 7, 2022 From roadtovr

Varjo, maker of high-end enterprise XR headsets, this week announced it has raised a $40 million Series D investment to continue building its Varjo Reality Cloud software and deliver a “true-to-life industrial metaverse.”

Varjo makes some of the highest-end enterprise XR headsets on the market, along with unique software solutions, claiming recently that one quarter of Fortune 100 companies have used used its tech.

This week the company announced it has raised $40 million in a Series D investment. Participants in the round include returning investors EQT Ventures, Atomico, Volvo Car Tech Fund, and Lifeline Ventures, and new investors Mirabaud and Foxconn, the latter being one of the world’s largest electronics manufacturers and a potential strategic partner for Varjo.

The leading investor in the round was not made clear. The raise brings Varjo’s total funding to $162.5 million since 2017, according to Crunchbase, and represents a down-round compared to the company’s Series C investment of $54 million in 2020.

“Our new funding is a testament to the incredible growth Varjo has seen over the past few years as interest for enterprise XR adoption grows,” said Timo Toikkanen, Varjo CEO. “The vision for a true-to-life metaverse for professionals is already here, and we are proud to be the first and only company in the world to continue to deliver human-eye resolution virtual and mixed reality technology to the largest and most iconic enterprises in the world.”

The so-called “metaverse for professionals” isn’t clearly defined by the company though it ostensibly refers to Varjo Reality Cloud, the company’s cloud-based XR streaming tech which aims to streamline the use of XR within large organizations. The company plans to expand the platform to a wider range of hardware and software, including headsets other than its own. Check out our exclusive preview of Varjo Reality Cloud earlier this year.

Alongside the funding announcement Varjo also named a new Chief Product Officer, Patrick Wyatt, who is said to be leading the company’s software and cloud projects.

Beyond its enterprise ambitions, Varjo also recently dipped a toe into the prosumer space with the release of its high-end Aero VR headset which, for the first time, could be bought by general consumers without any kind of monthly fee. The company’s Series D funding announcement didn’t offer any hints about the future of the Aero, but Varjo told us earlier this year that there’s a good chance of an eventual follow-up to the headset.

Meta & Qualcomm Join “Multi-year” XR Chip Partnership to Combat a Common Threat

September 3, 2022 From roadtovr

Two of the biggest names in XR—headset maker Meta and chip maker Qualcomm—today announced a “multi-year broad strategic agreement” to collaborate on “customized virtual reality chipsets” for future devices.

Qualcomm, a leading provider of smartphone processors, was an early mover in the XR space by pushing variants of its Snapdragon mobile processors as ideal for use in both AR and VR headsets—a play which now sees the company’s product in the vast majority of standalone headsets available on the market today.

Meta has used Qualcomm processors in all of its standalone headsets to date—Go, Quest, and Quest 2—and is expected to do the same in its forthcoming Project Cambria headset.

Today Meta and Qualcomm jointly announced they have entered into a “multi-year broad strategic agreement” to work together on XR platform development. The agreement was a big enough deal that the CEOs of both companies made the announcement together during the IFA 2022 conference.

“We’re working with Qualcomm Technologies on customized virtual reality chipsets— powered by Snapdragon XR platforms and technology—for our future roadmap of Quest products,” said Meta CEO Mark Zuckerberg. “As we continue to build more advanced capabilities and experiences for virtual and augmented reality, it has become more important to build specialized technologies to power our future VR headsets and other devices.”

With the companies having already worked together over the last several years, it’s a curious announcement—what gives?

On its face the announcement likely represents a commitment by Qualcomm to make Meta a top priority client over the next several years, devoting more time to the company and offering it more influence over future Snapdragon XR chips from Qualcomm. And with Meta believing that it’s going to take a complete rethinking of the typical computing architecture to make the sci-fi vision of XR a reality, the companies will probably be prototyping together on that front as well.

But there’s likely another major reason for this partnership—it brings two allies together against a common threat: Apple.

Though Apple hasn’t formally announced any XR products yet, all signs point to a long history of R&D and a desire for the company to dominate the space. For Meta, which itself wants to control the destiny of XR, that’s a problem. Mark Zuckerberg has been eyeing this potentiality since at least as far back as 2015, which drove him to buy Oculus in the first place—in an effort to get out ahead of companies like Apple and Google in the nascent XR space.

But Apple is a problem for Qualcomm too… Apple is sure to use its own custom processors (colloquially referred to as ‘Apple silicon’) in its XR products. By definition then, the greater marketshare that Apple has in the XR space, the fewer Snapdragon chips Qualcomm will sell.

Apple has long been building its own custom processors for its smartphones which has given the company and edge over competitors using commodity chips. In the last few years Apple has also begun phasing out third-party processors in favor of its own chips in its PC products, signaling a maturation of the company’s microprocessor design and fabrication capabilities.

For Meta, the partnership with Qualcomm buttresses a strategic vulnerability by giving the company a committed ally that can make chips that are highly specialized for XR devices.

For Qualcomm, the partnership with Meta is an effort to ensure that Apple doesn’t easily dominate the XR market and snuff out the company’s opportunity to sell chips to a wide variety of non-Apple XR device makers.

Ultimately the partnership is a maneuver in a fight for early ground in a market that the companies expect will one day be worth trillions of dollars.

Magic Leap Commits to OpenXR & WebXR Support Later This Year on ML2

June 7, 2022 From roadtovr

In an ongoing shift away from a somewhat proprietary development environment on its first headset, Magic Leap has committed to bringing OpenXR support to its Magic Leap 2 headset later this year.

Although Magic Leap 2 is clearly the successor to Magic Leap 1, the goal of the headsets are quite different. With the first headset the company attempted to court developers who would build entertainment and consumer-centric apps, and had its own ideas about how its ‘Lumin OS’ should handle apps and how they should be built.

After significant financial turmoil and then revival, the company emerged with new CEO and very different priorities for Magic Leap 2. Not only would the headset be clearly and unequivocally positioned for enterprise use-cases, the company also wants to make it much easier to build apps for the headset.

To that end Magic Leap’s VP of Product Marketing & Developer Programs, Lisa Watts, got on stage at week’s AWE 2022 to “announce and reaffirm to all of you and to the entire industry [Magic Leap’s] support for open standards, and making our platform very easy to develop for.”

In the session, which was co-hosted by Chair of the OpenXR Working Group, Brent Insko, Watts reiterated that Magic Leap 2 is built atop an “Android Open Source Project-based OS interface standard,” and showed a range of open and accessible tools that developers can currently use to build for the headset.

Toward the end of the year, Watts shared, the company expects Magic Leap 2 to also include support for OpenXR, Vulkan, and WebXR.

OpenXR is a royalty-free standard that aims to standardize the development of VR and AR applications, making hardware and software more interoperable. The standard has been in development since 2017 and is backed by virtually every major hardware, platform, and engine company in the VR industry, and a growing number AR players.

In theory, an AR app built to be OpenXR compliant should work on any OpenXR compliant headset—whether that be HoloLens 2 or Magic Leap 2—without any changes to the application.

OpenXR has picked up considerable steam in the VR space and is starting to see similar adoption momentum in the AR space, especially with one of the sector’s most visible companies, Magic Leap, on board.

Reality Labs Chief Scientist Outlines a New Compute Architecture for True AR Glasses

May 2, 2022 From roadtovr

Speaking at the IEDM conference late last year, Meta Reality Labs’ Chief Scientist Michael Abrash laid out the company’s analysis of how contemporary compute architectures will need to evolve to make possible the AR glasses of our sci-fi conceptualizations.

While there’s some AR ‘glasses’ on the market today, none of them are truly the size of a normal pair of glasses (even a bulky pair). The best AR headsets available today—the likes of HoloLens 2 and Magic Leap 2—are still closer to goggles than glasses and are too heavy to be worn all day (not to mention the looks you’d get from the crowd).

If we’re going to build AR glasses that are truly glasses-sized, with all-day battery life and the features needed for compelling AR experiences, it’s going to take require a “range of radical improvements—and in some cases paradigm shifts—in both hardware […] and software,” says Michael Abrash, Chief Scientist at Reality Labs, Meta’s XR organization.

That is to say: Meta doesn’t believe that its current technology—or anyone’s for that matter—is capable of delivering those sci-fi glasses that every AR concept video envisions.

But, the company thinks it knows where things need to head in order for that to happen.

Abrash, speaking at the IEDM 2021 conference late last year, laid out the case for a new compute architecture that could meet the needs of truly glasses-sized AR devices.

Follow the Power

The core reason to rethink how computing should be handled on these devices comes from a need to drastically reduce power consumption to meet battery life and heat requirements.

“How can we improve the power efficiency [of mobile computing devices] radically by a factor of 100 or even 1,000?” he asks. “That will require a deep system-level rethinking of the full stack, with end-to-end co-design of hardware and software. And the place to start that rethinking is by looking at where power is going today.”

To that end, Abrash laid out a graph comparing the power consumption of low-level computing operations.

As the chart highlights, the most energy intensive computing operations are in data transfer. And that doesn’t mean just wireless data transfer, but even transferring data from one chip inside the device to another. What’s more, the chart uses a logarithmic scale; according to the chart, transferring data to RAM uses 12,000 times the power of the base unit (which in this case is adding two numbers together).

Bringing it all together, the circular graphs on the right show that techniques essential to AR—SLAM and hand-tracking—use most of their power simply moving data to and from RAM.

“Clearly, for low power applications [such as in lightweight AR glasses], it is critical to reduce the amount of data transfer as much as possible,” says Abrash.

To make that happen, he says a new compute architecture will be required which—rather than shuffling large quantities of data between centralized computing hubs—more broadly distributes the computing operations across the system in order to minimize wasteful data transfer.

Compute Where You Least Expect It

A starting point for a distributed computing architecture, Abrash says, could begin with the many cameras that AR glasses need for sensing the world around the user. This would involve doing some preliminary computation on the camera sensor itself before sending only the most vital data across power hungry data transfer lanes.

To make that possible Abrash says it’ll take co-designed hardware and software, such that the hardware is designed with a specific algorithm in mind that is essentially hardwired into the camera sensor itself—allowing some operations to be taken care of before any data even leaves the sensor.

“The combination of requirements for lowest power, best requirements, and smallest possible form-factor, make XR sensors the new frontier in the image sensor industry,” Abrash says.

Continue on Page 2: Domain Specific Sensors »

Snap Acquires Brain-Computer Interface Startup NextMind

March 23, 2022 From roadtovr

Snap announced it’s acquired neurotech startup NextMind, a Paris-based company known for creating a $400 pint-sized brain-computer interface (BCI).

In a blog post, Snap says NextMind will help drive “long-term augmented reality research efforts within Snap Lab,” the company’s hardware team that’s currently building AR devices.

“Snap Lab’s programs explore possibilities for the future of the Snap Camera, including Spectacles. Spectacles are an evolving, iterative research and development project, and the latest generation is designed to support developers as they explore the technical bounds of augmented reality.”

Snap hasn’t detailed the terms or price of the NextMind acquisition, saying only that the team will continue to operate out of Paris, France. According to The Verge, NextMind will also be discontinuing production of its BCI.

Despite increasingly accurate and reliable hand and eye-tracking hardware, input methods for AR headsets still isn’t really a solved problem. It’s not certain whether NextMind’s tech, which is based on electroencephalogram (EEG), was the complete solution either.

NextMind’s BCI is non-invasive and slim enough to integrate into the strap of an XR headset, something that creators like Valve have been interested in for years. It’s also

Granted, there’s a scalp, connective tissue, and a skull to read through, which limits the kit’s imaging resolution, which allowed NextMind to do some basic inputs like simple UI interaction—very far off from the sort of ‘read/write’ capabilities that Elon Musk’s Neuralink is aiming for with its invasive brain implant.

Snap has been collecting more companies to help build out its next pair of AR glasses. In addition to NextMind, Snap acquired AR waveguide startup WaveOptics for over $500 million last May, and LCOS maker Compound Photonics in January.

Snap is getting close too. Its most recent Spectacles (fourth gen) include displays for real-time AR in addition to integrated voice recognition, optical hand tracking, and a side-mounted touchpad for UI selection.