In this new segment, we bring you the last week’s highlights from within the immersive tech industry.
First up, some news from Apple.
Apple has mostly been quiet in the XR tech sector. Despite repeated leaks and rumours about wearable AR devices in development, nothing official has been announced as yet. However Apple's flagship device, the iPhone continues to offer great support for AR experiences and the newest iteration, the iPhone 12, is no exception.
From an AR point of view, the most exciting news is the addition of a LIDAR scanner to the iPhone 12 Pro models.
Lidar is a highly accurate depth sensor that will allow the device to place AR objects more quickly and with greater accuracy. At the moment, placement of AR objects relies on the device generating point cloud data from the camera. While this method has come a long way, it can still get tripped up by certain surfaces and can take time to pick up enough data. With Lidar, the device can orient itself to its surroundings almost instantly, allowing for more stable AR experiences with improved occlusion of foreground objects, more in-depth interactions between real and virtual objects and improved object recognition. The Lidar scanner also adds a lot of functions useful to developers such as 3D environmental scanning.
Though the lidar scanner is only being added to the pro models this time round, this could very well be a sign of things to come. All in all leading to more powerful and immersive augmented reality experiences.
Next up, some more news from Facebook. If you watched the Facebook Connect Event back in September, some of you might have noticed footage of what seemed to be someone typing using the pass-through feature on the Quest 2 and an entirely virtual keyboard.
Well this week, Facebook have shed some light on what they’ve been up to. While the Quest 2 already has built-in hand tracking which lets users navigate the UI, browse the web and play a few supported games, the accuracy and responsiveness of the tracking has its limitations. To get around this, Facebook’s FRL researchers collected data using an external marker-based hand tracking system. This data was then fed into artificial intelligence to create a motion model which can predict what people intend to type, despite erratic movements or data missing from the less reliable hand tracking from a device such as the Quest.
According to Facebook, typists were able to average 73 words per minute using their hands, a flat surface and nothing else. This is similar in speed and accuracy to typing with a normal, physical keyboard. FRL says. “While still early in the research phase, this exploration illustrates the potential of hand tracking for productivity scenarios, like faster typing on any surface.”
Using the Quest 2 in business scenarios was a significant part of the message from the Connect event, so it will be really interesting to what new ideas and tools the research labs come up with.
And finally, an Augmented Reality experience bringing the natural world into your home. Rewild is a new experience from AR studio PHORIA. Produced in partnership with Netflix, Google and Oneplus, Rewild is based on the webby award-winning installation Rewild our Planet. It features content from the Netflix original documentary series and narration from everyone’s favourite naturalist, Sir David Attenborough. The experience will be delivered episodically, with each episode visiting one of Earth’s most precious biomes.
Users can create a playspace in their home to open a virtual screen and ‘rewild’ their environment in AR. The AR component showcases some of the latest in AR tech, making use of google’s AR-Core and Cloud Anchors, allowing the AR objects to persist in physical space.
The pilot episode, Frozen is available now on Google Play, with an iOS version coming to the App Store soon.