connect 2022

Meta’s Next-gen Avatar Presentation Was Too Good to Be True… For Now at Least

October 14, 2022 From roadtovr

This week Meta showed off a preview of its next-gen avatars during the Connect keynote. Frankly, they looked great! However, there’s… less than meets the eye here.

When Mark Zuckerberg appeared in Horizon during the Connect 2022 keynote sporting a next-gen Meta avatar with complete body tracking I was frankly blown away. Compared to the existing leg-less avatars that everyone appears as in Horizon, these new models not only had a full body, but also incredibly natural and movements with not a glitch in sight… it seemed almost too good to be true.

From what I know of the existing tech and limitations for tracking the body from a headset… what was shown here would have amounted to breakthrough work from Meta. Here was Zuckerberg literally jumping around and touching his feet with seemingly perfect tracking. His on-stage guest also highlighted what appeared to be perfectly accurate elbow tracking, something that’s only roughly estimated with today’s avatars. Zuckerberg then proceeded to say that these avatars would be launching sometime next year.

But all is not what it seems. After reaching out to Meta to clarify what tech was used to create that segment, a spokesperson told me the team used common external motion capture tech to drive the movement of the avatars, rather than its own solution running real-time on one of its headsets.

From my conversation with Meta, the company explained the segment was intended to show off how the next-gen Meta avatars look (from a model and texture standpoint) but how they moved was, “presented for illustrative, forward-looking purposes,” the company said.

So I have serious doubts that the first full-body VR avatars from Meta will have this level of high quality body tracking—and I’d argue the company was a little deceptive here.

What we’re probably going to see in the first iteration of Meta avatars with legs is an AI inference of leg positions that will look plausible from a third-person view… but nothing like being able to jump around and touch your feet—in fact, from a first-person view you probably won’t be shown your virtual legs.

But there’s no doubt that Meta’s avatars have been getting consistently better and that the company has been doing tons of R&D to try to reach the vision Zuckerberg presented.

Meta is burning the candle on both ends here. For one it’s steadily upgrading the avatars that actually exist today—like new face-tracking that will come thanks to Quest Pro—to make them more expressive and human. And on the other end, the company is working to try to make the most photorealistic avatars it can with its Codec Avatar project, which it hopes will one day be usable even on low-powered headsets.

So I won’t say that Meta will never have avatars that show the kind of full-body tracking quality that was shown off at Connect 2022. In fact there’s an interesting possibility that VR controllers with on-board cameras (like Touch Pro) might be able to solve this problem one day, but it seems certain this won’t be the case for the first version of Meta’s legged avatars.

Tech Demo Shows the Real Power of Quest Pro Face Tracking

October 12, 2022 From roadtovr

Quest Pro’s face-tracking capabilities will be quickly put to use to make Meta’s avatars more expressive, but next-gen avatars stand to benefit much more from the new tech.

One of Quest Pro’s big new features is a face-tracking system that uses internal cameras to sense the movement of your eyes and some parts of your face. Combined with a calibration-free machine learning model, the headset takes what it sees and turns it into inputs that can drive the animation of any avatar.

Key Quest Pro Coverage:

Quest Pro Revealed – Full Specs, Price, & Release Date

Quest Pro Hands-on – The Dawn of the Mixed Reality Headset Era

Quest Pro Technical Analysis – What’s Promising & What’s Not

Touch Pro Controllers Revealed – Also Compatible with Quest 2

In the near-term, this will be put to use with Meta’s existing avatars. And while it certainly makes them more expressive, they still look somewhat goofy.

This is likely the result of the current Meta avatar system not being built with this level of face-tracking in mind. The ‘rigging’—the underlying animation framework of the model—seems not quite fit for the task. Grafting Quest Pro’s face-tracking inputs onto the current system isn’t really doing justice to what it’s actually capable of.

Luckily Meta has built a tech demo which shows what’s possible when an avatar is designed with Quest Pro’s face-tracking in mind (and when almost all of the headset’s processing power is dedicated to rendering it).

Yes, it’s still a bit shaky, but every movement you’re seeing here is being driven by the user making the same motions, including things like puffing out the cheeks or moving the mouth from one side to the other. On the whole it’s a much more complete representation of a face that I’d argue manages to avoid entering into the uncanny valley.

I got to try this demo for myself in my recent hands-on with Quest Pro where I looked into the mirror and appeared as this character (which Meta calls Aura). I came away really impressed that, even with no special calibration, the face I saw in the mirror seemed to mimic whatever motions I could think to make with my face.

I was especially drawn to the detail in the skin. If I squinted and scrunched up my nose I could see the skin around it bunch up realistically, and the same thing when I raised my brow. These subtle details, like the crease in the cheeks moving with the mouth, really add a lot to the impression that this is not just an object in front of me, but something that’s got a living being behind it.

Whether or not the expressions actually look like me when I’m the one behind the mask is another question. Since this avatar’s face doesn’t match my own, it’s actually tough to say. But that the movements are at least plausibly realistic is a first important step toward virtual avatars that feel natural and believable.

Meta says it will release the Aura demo as an open source project so developers can see how they’ve attached the face-tracking inputs to the avatar. The company also says developers will be able to use a single toolset for driving humanoid avatars or non-human avatars like animals or monsters without needing to tweak every avatar individually.

Meta says developers will be able to tap a face-tracking API that uses values corresponding to FACS, a well recognized system for describing the movement of different muscles in the human face.

This is an effective system not only for representing faces, but it also forms a useful privacy barrier for users. According to Meta, developers can’t actually get access to the raw images of the user’s face. Insead they get a “series of zero-to-one values that correspond with a set of generic facial movements, like when you scrunch your nose or furrow your eyebrows,” Meta says. “These signals make it easy for a developer to preserve the semantic meaning of the player’s original movement when mapping signals from the Face Tracking API to their own character rig, whether their character is humanoid or even something more fantastical.”

Meta claims even the company itself can’t see the images captured by the headset’s cameras, either internal or external. They are processed on the headset and then immediately deleted, according to the company, without ever being sent to the cloud or to developers.

Microsoft & Meta to Bring Key Productivity Tools to Quest, Including Windows 11 via Cloud

October 11, 2022 From roadtovr

Meta and Microsoft today announced at Connect that starting in 2023 the Quest platform is getting a host of Windows productivity tools along with the ability to use Windows 11 via the cloud.

First, here’s a list of what’s coming to the Quest platform:

Microsoft Teams immersive meeting experiences for Meta Quest: Connect,
share, and collaborate in Teams immersive experiences.
Microsoft Windows 365: Stream the Windows experience on Quest Pro and Quest 2 devices, and access your personalized apps, content, and settings in VR.
Microsoft 365 app experiences: Interact with 2D content from
Sharepoint or productivity apps like Word, Excel, PowerPoint and Outlook directly from Quest Pro and Quest 2.
Microsoft Teams/Workrooms integration: Join a Teams meeting from inside
Workrooms.
Meta Avatars in Microsoft Teams: Use your Meta Avatar in Teams for
whiteboarding, brainstorming, and meetups.
Microsoft Intune and Azure Active Directory support: Enable
enterprise security and management on Quest Pro and Quest 2 devices.

Notably, Windows 365 gives business and enterprise users access to a version of Windows 10 or Windows 11, streaming from Cloud PCs to the user’s web browser.

The partnership is ostensibly building on Meta’s early steps towards virtual offices with Horizon Workrooms, something Meta says will help make Meta Quest Pro “an enterprise-ready device that’s easy to use, deploy, and manage at scale.”

Key Quest Pro Coverage:

Quest Pro Revealed – Full Specs, Price, & Release Date

Quest Pro Hands-on – The Dawn of the Mixed Reality Headset Era

Quest Pro Technical Analysis – What’s Promising & What’s Not

Touch Pro Controllers Revealed – Also Compatible with Quest 2

There’s no launch date in sight yet, however the companies say we can expect to see these apps sometime next year.

A new ‘Meta Quest for Business’ subscription bundle for Quest Pro and Quest 2 is also said to include “essential admin features” such as device and application management, premium support, and access to Microsoft Intune and Azure Active Directory.

“This means companies that want to provision Meta Quest devices can be confident that the security and management options they expect from PCs and mobile devices will be available in VR,” Meta says.

This comes alongside the official unveiling of Quest Pro, which carries with it the very business-centric price tag of $1,500 for the 256 GB model.

Meta Announces Four Accessories for Quest Pro, Including Light Blocker for Full Immersion

October 11, 2022 From roadtovr

Meta today officially unveiled Quest Pro, its high-end VR headset capable of AR interactions thanks to full-color passthrough. Since you’re shelling out $1,500 for the headset and controllers though, you might as well take a look around at what else the company has to offer.

Quest Pro already comes with a charging dock in the box; it’s an easier way of making sure your battery is always topped up—something you’ll need since Quest Pro’s battery only lasts between 1-2 hours.

Meta is selling a more compact charging dock for $80 though for easier on-the-go charging, available on October 25th. The two-in-one dock includes a 45W USB-C power adapter, capable of charging your Touch Pro controllers and any Quest headset with a USB-C connection.

Quest Pro has an open design that doesn’t fully block ambient light—fine for AR stuff, but not so fine for VR immersion. To compensate, Meta is selling a silicone ‘Full Light Blocker’, priced at $50 and out November 22nd. Here’s what it looks it looks like attached to the headset and without:

Quest Pro is a lot like Quest 2 in the audio department, providing the user an open-ear speaker. To let you shut out some of the outside world, Meta is pitching a pair of ‘VR Earphones’ for $50, out October 25th. Notice left/right independent design:

The headset has two 3.5mm jacks (left and right) so it’s very likely you can bring your own audio without issue, however we’re still awaiting confirmation on that.

If it’s anything like PSVR or the original Quest, both of which also feature similar left/right 3.5mm jacks, it will support a single pair of headphones plugged into one just fine.

Update: Both 3.5mm jacks are stereo capable, so you can easily bring your own audio.

And do you think you can just let a $1,500 device float around in your bag, making it vulnerable to scratches and inevitable fruit snack goo? Meta is also offering a Quest Pro Carry Case, priced at $120 and out October 25th.

Built by Incase, the carry case includes a custom-fit inner shell with interior pockets, securely storing your Quest Pro and other accessories.

You can check out all these over at Meta. Regardless of ship date, all will be available for pre-order starting today, October 11th.

Meta Connect Dev Conference to Be Held Virtually on October 11th

September 6, 2022 From roadtovr

Meta today announced that its next installment of Connect, the company’s annual XR developer conference, is again going digital this year. The event, which promises to share updates and looks into the “near and far future” of Meta, will be held on October 11th, 2022.

Like last year’s all-digital Connect, this year’s dev conference will include livestreaming keynotes and developer sessions, which the company says will be available on the Reality Labs Facebook page. And it’s all being boiled down into one day.

There’s no schedule out yet, however the company says its one-day virtual event will explore “the building of the metaverse and the future of augmented and virtual reality.” You can sign up for updates over at metaconnect.com for any upcoming livestreams.

We’d expect that much, but more specifically we’re hoping to learn a few key things. Topping the list is info surrounding its next VR headset, Project Cambria, or what is rumored to be dubbed Meta Quest Pro.

On the Joe Rogan Experience Podcast last week, Meta CEO Mark Zuckerberg let it slip that its “next device [is] coming out in October,” which suggests a Cambria launch during Connect. That would also mean we get the full info drop on the so-called Quest Pro then. Reminder: at more than $800 Cambria is likely meant for prosumers and developers looking to get their hands on a VR headset capable of AR interactions (aka ‘mixed reality’) thanks to color passthrough and built-in eye tracking.

Whatever we see though will most certainly need top last year’s big name change if the company wants to make good on its complete rebranding, which essentially unplugged the Facebook and Oculus brand names and pivoted the company to focus more on building the metaverse.

Some other bits we’d expect in Connect 2022: Meta’s social VR space Horizon Worlds demonstrating a more definite direction, which we’d hope includes better-looking avatars and the sort of baked-in social features its VR hardware has been missing since, well, forever.

Now that Oculus…er…Meta Quest 2 has seen a price bump and the company still hasn’t delivered on its ambitious metaverse concepts, Connect 2022 may be just the place for the company to set expectations for Meta moving forward.