Imagine strapping on a VR headset, blinding yourself to the real world, which you swap for a virtual world, and then navigating on foot to work? What could possibly go wrong?
Probably lots, but Microsoft reckons its DreamWalker VR system can let users do just this without walking into, say, a canal or oncoming traffic.
For those willing to place a bit of faith in technology, DreamWalker offers a solution if you would rather pretend you're walking in Manhattan rather than a dreary suburban landscape.
SEE: Executive's guide to the business value of VR and AR (free ebook)
Whether people would actually want to do this isn't the question. Microsoft researchers show that technically it is possible to do it with a fully immersive VR headset – as opposed to augmented reality – thanks to GPS, Windows Mixed Reality 'inside-out tracking', and two Kinect-like RGB depth sensors (RGBD) that help users avoid crashing into objects.
"The system adapts to a chosen route, making it possible to transform a walk to the grocery store or to a bus stop, say, into a walk through Times Square in VR," explains Microsoft.
The demo DreamWalker kit consists of a Samsung Odyssey headset, an HP Omen backpack VR PC, a Xiaomi Mi 8 for GPS capabilities, a battery pack, two Intel RealSense 425 RGB depth cameras, and a Windows Mixed Reality controller.
The project is a demonstration of how Microsoft is exploiting mobile hardware and input sensors to push the limits of 'spatial computing', a category that includes products like Oculus Rift, Microsoft HoloLens, and the Google-backed Magic Leap.
The researchers will be showing off the system this week in New Orleans at the ACM Symposium on User Interface Software and Technology (UIST) 2019. So the technology is part of Microsoft's exploration of new user interfaces that devices like the HoloLens could use in future.
VR is still in its early days but the price of headsets like Facebook's wireless Oculus Quest headset at $400 has some people excited about its future prospects.
Microsoft's researchers behind DreamWalker offer an upbeat prediction for the technology, even if sounds pretty grim for humans if VR takes off. In that future, toilet breaks might be the only reason a person detaches from the machine.
"We believe that future computer users will spend significantly more time in VR, where they remain immersed in entertainment, education, and work environments for long periods of time. It is entirely possible that future VR users may interrupt their VR sessions merely to accomplish real-world activities such as bathroom breaks or going to sleep," the researchers write.
As demonstrated in a video, the DreamWalker is guided through a virtual space by a yellow arrow that's informed by data from its sensors. The user would select a target destination on a map and then put on the VR headset to experience another location, such as downtown Manhattan, with traffic, other pedestrians, and the city's buildings.
"Behind the (VR) scenes, DreamWalker enables the user to reach their real-world destination through visual (re)direction at normal walking speeds, detecting obstacles along the way and presenting corresponding VR objects to prevent collisions," the researchers explain.
"DreamWalker fuses the data streams from two RGB depth cameras, optical inside-out tracking, and GPS position data in real time to detect the user's surrounding 3D obstacles and other people. DreamWalker tracks the user's 3D environment and continuously updates a map of walkable areas that is registered with real-world coordinates and landmarks, such as buildings and walkable paths."
Real obstacles are represented as virtual ones and the system predicts the path the user should take in alignment with the real world.
The project also attempts to address some of the limitations of augmented reality, such as Microsoft HoloLens, which allow users to see the real world through the augmented overlay, as well as games like Pokemon Go.
VR on the other hand isolates the user from the real world, with the drawback that it needs to be used indoors in a controlled environment.
DreamWalker builds on an older Microsoft system called VRoamer by detecting in real time minor obstacles such as street curbs while also tracking people standing and walking around the user.