Meta teams up with BMW for In-Car VR solutions

Meta and BMW say they’ve made a breakthrough in their car-based virtual reality project.

The two companies announced May 17th an update to their partnership, formed in 2021, to explore how augmented reality (AR) and virtual reality (VR) could be integrated into smart vehicles.

“VR headsets are equipped with a number of sensors but moving vehicles pose a tricky challenge, because tracking technology like Oculus Insight use both inertial motion sensors (IMUs) and cameras to precisely estimate the headset’s location and motion,” Meta said in a news release.

These modalities come into conflict in a moving environment, as cameras observe motion relative to the interior of the car, while the IMUs are focused on the “acceleration and rotational velocity relative to the world.”

To solve this problem, Meta worked with BMW to incorporate IMU data from a BMW vehicle’s sensor array in real time into Meta’s Project Aria research glasses’ tracking system, letting the system calculate the glasses’ location relative to the car. 

“That was a huge feat because after transferring the tracking system to a Meta Quest Pro, it allowed us to accurately anchor virtual objects to a moving car using a digital twin of the car,” Meta said. 

The company says it wants to continue working with BMW to leverage machine perception capabilities found in modern cars for additional uses.

“If we get it right, this technology could revolutionize travel in cars, trains, planes and beyond, unlocking new forms of hands-free communication, entertainment and utility,” Meta said.

“By working with developers like Niantic we are able to showcase new and creative ways for customers to discover relevant products beyond traditional online storefronts, enabling more immersive and engaging experiences both in-and-out of virtual worlds,” said Steve Downer, vice president of consumer electronics at Amazon.

Source