Arm unveils Mali-D77 display processor aimed at VR head-mounted displays

New hardware functions in Arm's next-generation DPU enable VR acceleration and help to eliminate motion sickness.

HP, HTC following the enterprise use cases with VR

Arm has announced its next-generation display processor, the Mali D77 -- formerly codenamed Mira. Like the existing Mali D71, the D77 is built on the Komeda architecture. The display processor's basic job is to support the GPU when handling complex workloads -- including HDR and virtual/augmented reality content, for example. As well as better performance, a DPU can deliver significant power savings, leading to longer battery life for mobile devices.

Arm's display processors, including the AD5 (Assertive Display) co-processor that primarily enhances outside viewability by dynamically adjusting screen brightness based on input from an ambient light sensor, are used in a variety of device types, ranging from PCs and tablets to VR head-mounted displays (HMDs).


The new Mali-D77 is aimed at premium smartphones likely to be used in VR holsters (such as Google's DayDream), and at VR head-mounted displays.

Image: Arm

The primary focus for the Mali-D77 is VR HMDs -- be they holsters for premium smartphones, wired or wireless companion devices, or standalone products.


Image: Arm

Up to four stereo VR layers can be composed by the Mali-D77, which is optimised for 3K resolution at 120 frames per second (fps), or 4K at 90fps. This should eliminate the motion sickness that has bedevilled earlier VR products; the new design should also allow device manufacturers to create smaller, lighter and more comfortable untethered VR headsets, Arm says.


Image: Arm

Mali-D77 enhancements

Three new hardware functions, all applied in a single pass through memory, have been added to the Mali-D77 to enable VR acceleration: Asynchronous TimeWarp (ATW); Lens Distortion Correction (LDC); and Chromatic Aberration Correction (CAC).


Image: Arm

Time-warping involves the reprojection of VR content to account for the user's head movements. "When the user is interacting with VR content with a head-mounted display on, what tends to happen is, they're going to move their head around, and so the content they're looking at is rotating, getting translated, getting transposed -- and all of this needs to happen in real time," explained Kushan Vyas, Arm's technical lead for the Mali-D77. "But the actual content, which was generated either by the graphics processor or by the video decoder, is rendered like a rectangle in the external memory. Taking that rectangle and processing it in real time -- that's the process of reprojection, and that's what the Mali-D77 does."

When it comes to distortion from the lenses in VR headsets, the Mali-D77 'predistorts' the inverse effect based on the characteristics of the lenses -- barrel distortion to compensate for pincushion distortion, for example. "With the Mali-D77, the lens properties are software-configurable, which allows it to be used with a wide range of head-mounted devices," said Vyas.

The Mali-D77 corrects for chromatic aberration in lenses using the same principle as it does for geometric distortion -- by applying the reverse of the defect. Again, the aberration will vary between lenses, which the D77 can handle in software.


The Mali-D77's advances over the previous-generation Mali-D71 are lens correction (distortion and chromatic aberration) and asynchronous timewarp (yellow boxes, above).

Image: Arm

When there's no DPU and VR processing occurs on the GPU, it typically takes between 10 and 20 percent of the cycles away from the GPU, said Vyas, along with an extra read/write pass through external memory. The result is higher perceived latency by the user, and lower pixel resolution and frame rate.

With DPU-based VR processing, resolution and frame rate can be maintained, while delivering savings of more than 180mW per VR layer and over 40 percent of DRAM bandwidth, Vyas said.


What Linus Torvalds really thinks about ARM processors

Lenovo launches AR/ VR headset for the enterprise

Hunting for concussions with VR

Facebook primes the Oculus Quest for business use cases

Zillow uses VR to try to make online real estate more authentic (CNET)