Google’s latest push into extended reality is taking shape. While the company isn’t entirely ready to show off any products just yet, it has laid out a vision for a unified Android XR ecosystem that will span a range of devices — such as virtual reality headsets and mixed reality glasses — in partnership with Samsung and Qualcomm.
This is evidently Alphabet’s latest attempt to compete with the likes of Meta and Apple on the extended reality front. The company has dabbled in this arena in the past with the likes of Google Glass, Daydream and Google Cardboard, programs that have found their way to the Google Graveyard. Android XR seems much more ambitious, and having some big-name partners on board from the jump indicates that Alphabet is much more serious about extended reality this time around.
Google has been beavering away on XR behind the scenes despite shutting down some of its higher-profile projects in that realm. “Google is not a stranger to this category,” Sameer Samat, president of Android Ecosystem at Google, told reporters ahead of the announcement. “We, like many others, have made some attempts here before. I think the vision was correct, but the technology wasn’t quite ready.”
One area where Google thinks that technology has advanced to the point where it’s ready to try again with XR is artificial intelligence. Gemini will be deeply integrated into Android XR. By tapping into the power of the chatbot and having a user interface based around voice and natural conversation, Google and its partners are aiming to deliver experiences that aren’t exactly possible to pull off using gestures and controllers.
“We are fully in what we refer to as the Gemini Era, and the breakthroughs in AI with muti-modal models are giving all of us totally new ways of interacting with computers,” Samat said. “We believe a digital assistant integrated with your XR experience is the killer app for the form factor, like what email or texting was for the smartphone.”
Google believes that smart glasses and headsets are a more natural form factor to explore this tech with, rather than holding up your smartphone to something in the world that you want Gemini to take a look at. To that end, the wide array of XR devices that are popping up, such as VR headsets with passthrough (the ability to see the outside world while wearing one) is another factor in Google’s push into that space.
We’ll get our first real look at Android XR products next year, including one that Google is developing in partnership with Samsung. The first headset, currently dubbed Project Moohan (which means “infinity” in Korean), will feature “state-of-the-art displays,” passthrough and natural multi-modal input, according to Samsung. It’s slated to be a lightweight headset that’s ergonomically designed to maximize comfort.
Renderings of the Moohan prototype (pictured above) suggest the headset will look a little like the Apple Vision Pro, perhaps with a glass visor on the front. Along with the headset, Samsung is working on Google XR glasses, with more details to come soon.
But nailing the hardware won’t matter much if you can’t do anything interesting with it. As such, Google is now looking to bring developers into the fold to create apps and products for Android XR. The company is offering developers APIs, an emulator and hardware development kits to help them build out XR experiences.
On its side of things, Google is promising an “infinite desktop” for those using the platform for productivity. Its core apps are being reimagined for extended reality as well. Those include Chrome, Photos, Meet, Maps (with an immersive view of landmarks) and Google Play. On top of that, mobile and tablet apps from Google Play are said to work out of the box.
On YouTube, it looks like you’ll be able to easily transition from augmented reality into a VR experience. And in Google TV, you’ll be able to switch from an AR view to a virtual home movie theater when you start a film.
A demo video showed a headset wearer using a combination of their voice and a physical keyboard and mouse to navigate a series of Chrome windows. Circle to Search will be one of the many features. After you’ve used the tool to look up something, you can use a Gemini command to refine the results. It’ll be possible to pull 3D image renderings from image search results and manipulate them with gestures.
As for AR glasses — essentially next-gen Google Glass — it seems that you’ll be able to use those to translate signage and speech, then ask Gemini questions about the details of, say, a restaurant menu. Other use cases include advice on how to position shelves on a wall (and perhaps asking Gemini to help you find a tool you put down somewhere), getting directions to a store and summarizing group chats while you’re on the go.
Thanks to advances in technology, AR glasses look much like regular spectacles these days, as we’ve seen from the likes of Meta and Snap. That should help Google avoid the whole “Glass-holes” discourse this time around given that there shouldn’t be an obscenely obvious camera attached to the front. But the advancements might give cause for concern when it comes to privacy and letting those caught in the camera’s cone of vision know that they’re perhaps being filmed.
Privacy is an important consideration for Android XR. Google says it’s building new privacy controls for Gemini on the platform. More details about those will be revealed next year.
Meanwhile, games could play a major factor in the success of Android XR. They’re a focus for Meta’s Quest headsets, of course. On the heels of its various missteps with Stadia, Google is hoping to make it as easy as possible for developers to port their games to its ecosystem.
Not only that, Unity is one of the companies that’s supporting Android XR. Developers will be able to create experiences for it using the engine. Unity says it will offer full support for Android XR, including documentation and optimizations to help devs get started. They can do that now in public experimental versions of Unity 6.
Resolution Games (Demeo) and Google’s own Owlchemy Labs (Job Simulator) are among the studios that plan to bring titles built in Unity to Android XR. The process is said to be straightforward. “This is as simple a port as you’re ever going to encounter,” Owlchemy Labs CEO Andrew Eiche said in a statement.
Meanwhile, Unity has teamed up with Google and film director Doug Liman’s studio 30 Ninjas to make a “new and innovative immersive film app that will combine AI and XR to redefine the cinematic experience.”
Since gaming is set to play a sizable role in Android XR, it stands to reason that physical controllers will still be a part of the ecosystem. Not many people are going to want to play games using their voice.
But that’s the key: Android XR is shaping up to be a broad ecosystem of devices, not just one. This strategy has paid dividends for Google, given the spectrum of phones, tablets, cars and TVs that variants of Android are available on. It will be hoping to replicate that success with Android XR.
