Whether people would actually want to do this isn't the question. Microsoft researchers show that technically it is possible to do it with a fully immersive VR headset – as opposed to augmented reality – thanks to GPS, Windows Mixed Reality 'inside-out tracking', and two Kinect-like RGB depth sensors (RGBD) that help users avoid crashing into objects.
"The system adapts to a chosen route, making it possible to transform a walk to the grocery store or to a bus stop, say, into a walk through Times Square in VR," explains Microsoft.
The demo DreamWalker kit consists of a Samsung Odyssey headset, an HP Omen backpack VR PC, a Xiaomi Mi 8 for GPS capabilities, a battery pack, two Intel RealSense 425 RGB depth cameras, and a Windows Mixed Reality controller.
The project is a demonstration of how Microsoft is exploiting mobile hardware and input sensors to push the limits of 'spatial computing', a category that includes products like Oculus Rift, Microsoft HoloLens, and the Google-backed Magic Leap.
Microsoft's researchers behind DreamWalker offer an upbeat prediction for the technology, even if sounds pretty grim for humans if VR takes off. In that future, toilet breaks might be the only reason a person detaches from the machine.
"We believe that future computer users will spend significantly more time in VR, where they remain immersed in entertainment, education, and work environments for long periods of time. It is entirely possible that future VR users may interrupt their VR sessions merely to accomplish real-world activities such as bathroom breaks or going to sleep," the researchers write.
As demonstrated in a video, the DreamWalker is guided through a virtual space by a yellow arrow that's informed by data from its sensors. The user would select a target destination on a map and then put on the VR headset to experience another location, such as downtown Manhattan, with traffic, other pedestrians, and the city's buildings.
"Behind the (VR) scenes, DreamWalker enables the user to reach their real-world destination through visual (re)direction at normal walking speeds, detecting obstacles along the way and presenting corresponding VR objects to prevent collisions," the researchers explain.
"DreamWalker fuses the data streams from two RGB depth cameras, optical inside-out tracking, and GPS position data in real time to detect the user's surrounding 3D obstacles and other people. DreamWalker tracks the user's 3D environment and continuously updates a map of walkable areas that is registered with real-world coordinates and landmarks, such as buildings and walkable paths."
Real obstacles are represented as virtual ones and the system predicts the path the user should take in alignment with the real world.
The project also attempts to address some of the limitations of augmented reality, such as Microsoft HoloLens, which allow users to see the real world through the augmented overlay, as well as games like Pokemon Go.
VR on the other hand isolates the user from the real world, with the drawback that it needs to be used indoors in a controlled environment.
DreamWalker builds on an older Microsoft system called VRoamer by detecting in real time minor obstacles such as street curbs while also tracking people standing and walking around the user.