That is the question of the "black box" of AI that has been much debated in recent years. This week, the machine learning scientists at Nvidia report some progress in understanding what's going on.
In a bravura display of fake facial images, the researchers claim a novel way to separate out the aspects of pictures that the neural net "sees" at a high level, such as the orientation of an object, and at a low level, such as details of texture.
Using a technique called "adaptive instance normalization," or "AdaIN," introduced last year by Cornell University researchers Xun Huang Serge Belongie, the high-level and low level features can be extracted from each image to create a style.
The Nvidia team added a twist: they can manipulate the different levels of features, high-level to low-level freely, a much more nimble way to mix and match the properties of the faces, from head size on down to the freckles.
By tuning the images in such a way, the theoretical payoff is that it's clearer what the network is doing at each instance of its process, a kind of window into its operation.
The practical effect is the ability to rapidly, effortlessly "morph" fake headshots by adjusting controls in software as easily as you would change the color of a picture in Photoshop, which you can see demonstrated in the video.
A striking additional discovery is that the GAN they created is now working on much less information than past fakes. Rather than laboriously "mapping" from pixels of one image to another to transfer styles, it's using only the style cues it gets from the AdaIN.
As the authors write, "We find it quite remarkable that the synthesis network is able to produce meaningful results even though it receives input only through the styles that control the AdaIN operations."
Think of it as the Mr. Potato Head of headshots. Fakes will never be the same again.
Innovative artificial intelligence, machine learning projects to watch