google

Google’s Project Starline Starts Real-world Testing, Bringing Light-field Video Calls to Partners

October 14, 2022 From roadtovr

Google’s Project Starline is an experimental system for immersive video chatting that aims to close the distance between people without needing an AR or VR headset. At Google’s annual Cloud Next conference, the company announced it’s rolling out the booth-sized device in early access to select enterprise partners, officially taking it beyond the walls of Google’s offices for the first time.

The company says in a blogpost that early access to Project Starline is starting this year, with the installation of prototypes at the offices of Salesforce, WeWork, T-Mobile, and Hackensack Meridian Health.

Project Starline provides a glasses-free 3D chatting experience thanks to its host of sensors, light-field display, spatial audio, computer vision, and novel compression to make the whole experience possible over the web. The end effect is you get to have a natural face-to-face conversation with another person, including eye contact.

Starline prototypes have already been in use in Google offices across the US since late 2021, something the company says aims to increase employee presence, attentiveness, and productivity compared to traditional video calling solutions. Since then, Google says it’s showed off the tech to 100 enterprise partners in areas such as media, healthcare, and retail to see where it needs to improve.

“As we build the future of hybrid work together with our enterprise partners, we look forward to seeing how Project Starline can help employees form strong ties with one another, doctors form meaningful bonds with their patients, and salespeople make deeper connections with their clients and customers,” says Andrew Nartker, Starline’s Director of Product Management. “Whether you’re presenting to a colleague or just sitting down for a coffee chat, we want the Project Starline experience to feel natural, as if the person is sitting in the same room as you. More broadly, we are eager to enable workforces to feel energized and productive when collaborating from afar.”

The company says it will share more about its early access program at some point next year, as it no doubt aims to further tailor Starline into a shippable product.

Google to Publicly Test AR Prototypes Starting in August

July 20, 2022 From roadtovr

Google announced that the company will be conducting real world tests of its early AR prototypes starting next month.

The company says in a blog post that it plans to to test AR prototypes in the real world as a way to “better understand how these devices can help people in their everyday lives.”

Some of the key areas Google is emphasizing are things like real-time translation and AR turn-by-turn navigation.

“We’ll begin small-scale testing in public settings with AR prototypes worn by a few dozen Googlers and select trusted testers,” the company says. “These prototypes will include in-lens displays, microphones and cameras — but they’ll have strict limitations on what they can do. For example, our AR prototypes don’t support photography and videography, though image data will be used to enable experiences like translating the menu in front of you or showing you directions to a nearby coffee shop.”

Critically, Google says the research prototypes look like “normal glasses.” This was no doubt partially informed by their rocky experience with Google Glass starting in 2013 which spawned the neologism ‘glasshole’ due to the device’s relative high visibility and supposed privacy concerns of wearing a camera. Glass is still around, albeit only for enterprise users.

Google says it wants to take it slow with its AR glasses though and include a “strong focus on ensuring the privacy of the testers and those around them.” Although the units will clearly pack camera sensors to do its job, Google says after translating text or doing turn-by-turn directions, the image data will be deleted unless it will be used for analysis and debugging.

“In that case, the image data is first scrubbed for sensitive content, including faces and license plates. Then it is stored on a secure server, with limited access by a small number of Googlers for analysis and debugging. After 30 days, it is deleted,” the company says in a FAQ on the program.

Testers will also be prohibited from testing in public places such as schools, government buildings, healthcare locations, places of worship, social service locations, areas meant for children (e.g., schools and playgrounds), emergency response locations, rallies or protests, and other similar places. For navigation, testers are also banned from using AR prototypes while driving, operating heavy machinery, and engaging in sports.

Google’s inclusion of displays in its public prototypes is a step beyond Meta’s Project Aria, which started on-campus testing of AR prototypes in 2020 that notably included everything you’d expect from AR glasses but the displays. We’re waiting to hear more about Meta’s Project Nazare however, which are said to be “true augmented reality glasses.”

As for Apple, well, there’s only rumors out there for now on specifications and target launch dates for the company’s MR headset and follow-up AR glasses. It’s clear however we’re inching ever closer to a future where the biggest names in established tech will directly compete to become leading forces in what many have hailed as a class of device which will eventually replace your smartphone.

Google Acquires MicroLED Startup Raxium to Bolster AR Ambitions

May 5, 2022 From roadtovr

Google is adding to its portfolio of XR microdisplay designs and talent, as the company announced it’s acquired microLED (µLED) designer Raxium.

The acquisition was previously reported by The Information in March, however now Google has confirmed in a blog post that it has indeed acquired Raxium, a five-year old startup building microdisplays for use in AR and VR headsets.

The Information’s report held Raxium was sold to Google for $1 billion, however official details of the acquisition are still murky. Google says Raxium will join its Devices & Services team, which is tasked with development of Google’s consumer devices.

It’s thought that Raxium will allow for Google to create lighter, cheaper displays for its upcoming AR devices. While conventional Super AMOLEDs found in smartphones measure around 50 µm per pixel, Raxium says it’s shrunk its microdisplays to feature µLED measuring 3.5 µm per pixel. The company claims its technology has led to an efficiency “5X greater than the previously published world record.”

“The team at Raxium has spent five years creating miniaturized, cost-effective and energy efficient high-resolution displays that have laid the foundation for future display technologies. Raxium’s technical expertise in this area will play a key role as we continue to invest in our hardware efforts,” says Rick Osterloh, senior VP of Google’s Devices & Services team.

Google is undoubtedly gearing up to release XR headsets of some type in the future, which may compete with devices from Apple, Meta, Microsoft and Snap. In 2020, Google acquired North, a Canada-based company known for creating unobtrusive prescription-compatible smartglasses—a sight more stylish than Google Glass. Check out our primer on the difference between smartglasses and AR headsets to learn more.

Earlier this year Google snapped up Bernard Kress, principal optical architect on the Microsoft HoloLens team. Kress is now the Director of XR Engineering at Google Labs, an internal XR division founded late last year. According to previous reports, Google Labs is currently working on an AR headset, code named Project Iris, which is rumored to ship sometime in 2024.

Reports detailing Project Iris maintain a device providing a standalone experience with onboard power, computing, and outward-facing cameras for world sensing capabilities—similar in description and function to headsets like HoloLens or Magic Leap.

HoloLens Optics Chief Joins Google Amid Reported Push for Upcoming Google AR Headset

January 21, 2022 From roadtovr

Bernard Kress, principal optical architect on Microsoft’s HoloLens team, has left the company to take on the role of Director of XR Engineering at the recently formed Google Labs. A report by The Verge maintains Google is also now gearing up to produce an AR headset that could directly compete with similar offerings from the likes of Apple and Meta.

Before joining Microsoft in 2015, Kress worked as principal optical architect behind Google Glass, the company’s smartglasses that found marked success in the enterprise sector after a rocky reception by consumers in 2013.

At Microsoft, Kress continued his work—principally focused on micro-optics, wafer scale optics, holography and nanophotonics—as partner optical architect on the HoloLens team, overseeing the release of both HoloLens and HoloLens 2.

Now Kress is back at Mountain View working on Google’s next AR headset. According to his LinkedIn, Kress has been leading the Optical Engineering department at Google Labs since November 2021—or right as Google shook things up by creating the AR/VR division.

And there’s no doubts about it: Kress says he’s focusing on creating consumer AR hardware at Google.

Hot on the heels of the strategic hire, a report from The Verge maintains Google is now gearing up to produce its own AR headset, which is allegedly codenamed Project Iris.

According to people familiar with the matter, Project Iris is said to ship sometime in 2024, although that date may simply be wishful thinking given the early stage of the project.

The prototype is said to be ski goggle-esque, providing a standalone experience with onboard power, computing, and outward-facing cameras for world sensing capabilities—similar in description and function to headsets like HoloLens or Magic Leap.

The standalone AR headset is said to use a custom Google processor running on either a version of Android or Google’s own Augmented Reality OS, which according to a recent job listing is currently in development.

Around 300 people are purportedly working on Project Iris, however Google plans to expand by “hundreds more.” Veteran AR/VR Google exec Clay Bavor is heading up the project, reporting directly to CEO Sundar Pichai.

Bavor is known for his work on Project Starline, an experimental light field display system created to be more natural way of chatting at a distance than conventional video conferencing apps. Bavor also oversaw the 2016 launch of Google’s Daydream VR platform (subsequently abandoned in 2019), and the development of ARCore, the software development kit for smartphone-based AR.

This comes as Apple is supposedly preparing to release a VR headset with passthrough AR capabilities (sometimes called ‘mixed reality), which reports maintain will come at some point in 2023 as a precursor to a dedicated Apple AR headset at some point afterwards.

Meta (formerly Facebook) is also working on its own VR headset with AR passthrough, codenamed Project Cambria, which may be positioned as direct competition to Apple’s own when the time comes.

Google Reportedly Building AR Headset Codenamed Project Iris

January 21, 2022 From vrfocus

Google has quite the chuequered history when it comes to virtual reality (VR) and augmented reality (AR) devices. Most have failed to gain any momentum like the original Google Glass or Daydream View – Google Cardboard had it time in the sun though – but it seems the company hasn’t dropped those hardware aspirations. A new report this week suggests that Google is working on a new AR device codenamed Project Iris.

Google Glass Enterprise Edition2 — Google Glass Enterprise Edition 2 uses Kopin’s nHD Display. Image credit Kopin

As reported by The Verge, Project Iris is an AR headset in early development that sports outward-facing cameras to supply video of the real world that can then be combined with computer graphics for a mixed reality experience. There’s a suggestion that it’ll be more immersive than Magic Leap.

The prototype is rumoured to currently resemble a pair of ski goggles and that the device will be entirely standalone, requiring no external power source. That puts it more in line with Microsoft’s HoloLens 2 device rather than Magic Leap 2 or Nreal Light which require an external processor puck or smartphone respectively.

It’s powered by a custom Google processor, runs on Android but if recent job listings are anything to go by it’ll have its own unique UI. There’s also the indication that the AR device will support cloud streaming, utilising Google’s data centres to help improve the fidelity. That’s to be expected as it means the onboard processor doesn’t need to be hugely powerful. Cloud streaming has been accelerating in the XR space, last year Google Cloud revealed a collaboration with NVIDIA CloudXR and Varjo announced the feature for its Reality Cloud platform this week.

How Project Iris will stack up against more vocal tech giants in the XR space such as Meta remains to be seen but just like Apple and its secretive plans, Google is trying to keep the project a secret. Supposedly the Pixel team are involved and the core team involved in Iris numbers around 300 people with more being hired, all overseen by Google VR boss Clay Bavor.

The sources familiar with Project Iris claim that 2024 has been earmarked for its launch. For continued updates on Project Iris, keep reading gmw3.