CES 2019: Nvidia CEO Huang explains how AI changes everything

Nvidia's chief executive, Jensen Huang, took to the stage of the ballroom at the MGM Grand hotel in Las Vegas on Sunday night, the opening night of the Consumer Electronics Show, to tell those assembled that AI, especially deep learning, is fundamentally changing his company's business of creating lifelike computer graphics.

Also: CES 2019: The best tech, gadgets on show (so far)

The traditional graphics pipeline is yielding to neural network approaches, accelerated by newer on-chip circuitry, so that physics simulation and sampling of real-world details are taking over from the traditional practice of painting polygons on the screen to simulate objects and their environment. 

Huang pointed to how primitive a lot of graphics still looks, saying that "in the last 15 years, technology has evolved tremendously, but it still looks largely like a cartoon."

Also: Nvidia AI research points to an evolution of the chip business 

At the core of computer graphics today is the process of rasterization, whereby objects are rendered as collections or triangles. It's a struggle to convincingly employ rasters for complex nuances of light and shadow, Huang noted.

Three things, he said, are missing: "Reflections aren't right, shadows aren't right, and refractions are really hard to do," said Huang. To remedy that, one technology the company is pushing is ray tracing, where the computer models the physics of photons interacting with the world.

"It's hard to simulate the effects of light from geometry," meaning, trying to "paint" light onto all the raster triangles, said Huang. Trying to "bake" the light into those triangles hasn't worked that well despite ingenious attempts. Instead, "you have to start from the light, tracing light from your eyes into the world." That ray tracing technology has been around for decades but it has not been fast enough to create light effects in real time. "It took ten years to figure out how to do ray tracing fast enough," said Huang, "and it wouldn't have been possible without deep learning."

To make ray tracing render stunning effects such as umbra and penumbra, the nuances of shadows, and reflections on glass and in water, the workload is split between the physics model and a neural network approach that the company calls "deep learning super sample, or "DLSS." Nvidia has said the approach uses an autoencoder style of neural network to infer sixty-four samples per pixel in every pixel of every image in a training set of rendered images. By performing this sampling, the network learns a way to apply superior anti-aliasing to images.  

"DLSS predicts the perfect pixel, it takes in a lower-resolution image and outputs a higher-resolution one."  

Also: Intel's AI chief sees opportunity for 'massive' share gains

This reliance on AI to assist rendering is something ZDNet pointed out in an interview last month with Nvidia's Bryan Catanzaro, who is the head of applied deep learning research at the company. More and more, "model-based" programming is taking over the traditional graphics programming business, meaning, the use of a neural network model to infer the look of scenes rather than a person programming rules of polygons to assemble the scene. 

Combining ray tracing with DLSS is a form of hybrid computing, which the company has branded "RTX."  While DLSS is trained on a supercomputer made up of the company's "DGX2" chips, the real-time rendering is performed in the client device by plug-in accelerator cards. One example is the "GeForce RTX 2060," a new plug-in card that Huang revealed at the show that will retail this month for $349, as ZDNet's Stephanie Condon related last night. 

The 2060 splits the work of ray tracing and DLSS across two separate kinds of processor elements in the GPU, the one being the "RT" cores, for ray tracing, the other being "tensor" cores that perform the DLSS inference that fills in frames. The company argues that the RTX technology is able to achieve an optimal balance in the two forms of compute, on top of the standard rendering work of rasterization, to achieve better images without slowing down the rate of frames sent to the screen. 

Huang showed off multiple games that will be shipping this year that will take advantage of the split-computing approach, including EA's Battlefield V, BioWare's Anthem, and the "alt-universe first-person shooter" Atomic Heart from Moscow studio Mundfish. He called the attention of the audience to details in each of the games, punctuating his pitch with frequent reminders that "this is not a movie," meaning that the frames of graphics are being rendered in real time between the RT and tensor circuitry. "It's fantastic!" is among the many triumphant exhortations that Huang exclaimed during the demos, as he had the staff turn on and off the RTX capabilities to show the difference with and without the hybrid assistance of AI. 

Must read

• CES 2019 Las Vegas CNET
  
• CES 2019: A first look at the cool tech TechRepublic
  

Huang sees a much broader role for the hybrid computing than just games. He pointed out the worldwide community of millions of gamers who have interests that dovetail with gaming such as animation. That global community are creating some of the new artworks of the age, he said. Nvidia is partnering with the video camera makers RED Digital Cinema to make possible editing of 8K video on laptops with the RTX technology. 

Writ broadly, "AI and ray tracing are two fundamental pieces of technology that we think will define the next generation of graphics," said Huang. 

CES 2019: Weird and wonderful tech, gadgets

More CES 2019 coverage:

• CES 2019: Ford demos cellular V2X with Qualcomm chipset
  
• CES 2019: Nvidia's new GeForce RTX 2060 is just $349
  
• CES 2019: Voice activated trash disposal
  
• CES 2019: What to expect from the chipmakers
  
• CES 2019: HP, Acer, and Asus unveil new laptops
  
• CES 2019: 5G, AI, design and data collide
  
• CES 2019: Robotic suitcases back (and ... maybe better?)