X
Innovation

Facebook publishes new 360 video standards

Called SSIM360 and 360QVM, the new standards address the warping and field-of-view problems that arise when trying to assess 360 content.
Written by Stephanie Condon, Senior Writer

Video: Fearing scams, Facebook bans cryptocurrency ads

Facebook on Friday published two new standards for 360 video, designed to resolve the quality assessment challenges presented by 360 content.

Specifically, the new standards, called SSIM360 and 360QVM, address the warping and field-of-view (FOV) problems that arise when trying to assess 360 content. SSIM360 is used to assess the quality of individual encoding and playback components, while 360QVM assesses the overall quality of a streaming session.

Read also: Facebook updates Messenger with more business chat features

Facebook VR research scientist Shannon Chen explained in a blog post why Facebook developed the QA standards, which it hopes will be adopted across the industry:

"With every new technology, it's important to be able to assess the quality of the immersive experience compared with previous approaches in terms of things like resolution, bitrate, or structural similarity," he wrote. "Unfortunately, there isn't a broadly accepted QA standard for 360 content, and the established QA workflows for conventional, non-360 media don't capture the relevant properties of a 360 video -- immersiveness and viewer control.

Why are the new standards needed?

To understand why new standards are needed, you have to first understand the current standards. Currently, image and video encodings are typically evaluated with the Structural Similarity Index (SSIM). SSIM is a score based on how closely encoded content matches the orginal content. For video, a frame-by-frame assessment is conducted. For each frame, SSIM evaluates smaller segments. Each segment (like the squares below) is given an SSIM score. These are averaged to produce an SSIM score for each frame. For a video, each frame's SSIM score is averaged.

Read also: Facebook patches admin information leak vulnerability

fb3601.png
(Image: Facebook)

To produce an SSIM score, each segment is static and has an identical weight in the final averaging. However, this process isn't reliable for 360 content.

To illustrate why, consider the two maps below, which are both encoded with a codec that degrades the portion of the map where you see Antarctica. The maps should look the same when rendered into 3D spheres. However, the SSIM process would give the map on the left a lower score because warping is not considered, and the degraded portion takes up a larger proportion of the map.

Read also: BlackBerry sues Facebook for alleged patent infringement

fb3602.png
(Image: Facebook)

Solving the problem with SSIM360

To solve this problem, Facebook created SSIM360. This is measured by putting a weight on each per-segment SSIM score when they're averaged together. The more a sampled area is stretched, the smaller the weight, Chen explains. Since each sample contributes differently to the final score, it effectively cancels out the warping effect.

Chen explains how Facebook verifies the outcome of this technique, concluding that SSIM360 reduces error by about 50 percent.

How 360VQM comes into play

However, SSIM360 becomes less valuable when one considers that viewers of 360 content only have about 15 percent of the content within their FOV at any given time. Since the SSIM360 score represents the entire frame -- rather than the parts of the frame where the viewer is looking -- it doesn't adequately measure the final view quality.

Furthermore, it doesn't consider the sort of view-dependent optimizations used to deliver 360 content, such as offset projections, saliency-based encoding, and content-dependent streaming. These techniques allocate resources to optimize parts of the frame where the viewer is most likely to look.

Read also: The five things that kill your iPhone's battery the fastest

fb3603.png
(Image: Facebook)


Facebook solves this with two mappings. The first mapping, if you consider the image above, effectively measures the number of pixels in the red circle. The second mapping applies a scaling factor to the SSIM360 score to reflect the change in pixel density.

This creates the 360VQM standard (360 video quality metric). "It accurately and efficiently captures the quality change in the encoding process, and imposes the effect of resolution change during playback onto the final score to account for view-dependent optimizations," Chen explained.

fb3604.png
(Image: Facebook)

Previous and related coverage

Google spinoff Waymo's 360-degree video: See what it's like to be a self-driving car

Waymo releases video to assure people its autonomous-vehicle technology is safe.

Samsung's new 360-degree camera has 17 lenses for VR streaming

360 Round is aimed at specialist VR content creators.

Editorial standards