Apple eyes sensor-packed augmented reality headset
Apple is rarely first to launch entirely new products, but it's got a knack for sweeping up entire categories by improving on first-mover tech as it did with the iPod, iPhone, and Apple Watch.
Innovation
Its next target is likely to be devices like Microsoft's HoloLens and Facebook's Oculus Rift headsets. As ZDNet sister site CNET reported last year, Apple has long been believed to have a headset in the works capable of augmented reality (AR) and virtual reality (VR), and which could be released as soon as 2020.
The company has revealed some of the ideas it's been exploring in a newly-filed US patent application for sensors on an Apple head-mounted display that captures the wearer's head pose, surrounding light, gaze, hand gestures, and facial expressions including the user's eyebrows, mouth, and jaw movements.
It would be combined with multiple "world-facing" or outward-facing sensors that would collect information about the user's environment, including depth information, and then render those images in the user's display.
SEE: Mixed reality in business (ZDNet/TechRepublic special feature) | Download the free PDF version (TechRepublic)
The idea would be to represent the wearer's body, gestures and facial expressions in a 3D view they'd see through the headset's viewer. Or, in Apple's words, it would use the information to "render avatar of the user for display in the 3D virtual view based at least in part on information collected by the plurality of user-facing sensors" and then blend that with information captured from world-facing sensors.
As pointed out by Variety, which first spotted the patent, Apple's idea to capture the real-world, and then and show representations of it on a display, is notably different from that taken by Microsoft's HoloLens and Magic Leap. Apple's approach would capture real-world images and then render them on a display.
Additionally, the focus on sensors for capturing and rendering facial expressions would translate well to a headset using Apple's existing Animoji and Memoji avatars on the latest iPhones.
Free download: Top 20 Apple keyboard shortcuts for business users TechRepublic
While it remains to be seen what design choices Apple will make if it releases a headset, the current patent suggests it will be packed with sensors, some of which are drawn from camera setup for FaceID in the iPhone X, XS, and XR.
Apple's illustrations indicate a dedicated sensor for each eyebrow, two lower jaw sensors, multiple downward-facing infrared hand sensors, one or more eye-tracking sensors, two cameras for capturing video, and around three world sensors, which could include a LIDAR sensor for depth information.
Apple is allowing for the controller to be either part of the headset or an external computing device, like an iPhone, that's connected to the headset.