What should the world look like? Some neural nets have their own answer, find researchers
New research from MIT is shedding light on the mysterious inner workings of Generative Adversarial Networks (GANs), and can reveal how the algorithms make chillingly human-like decisions about the way they think the world should look like.
An unsupervised form of machine learning, a GAN can generate hyper-realistic new audio or video content based on a dataset of examples. The fabricated content created by the technology is notoriously difficult to distinguish from authentic images and sounds.
Artificial Intelligence
GANs, in fact, are the secret recipe behind deepfakes – synthetic media that uses fake audio and video to impersonate another person, in many cases to discredit a public figure. Often used as an example is a fake video of Nancy Pelosi released last year, in which the US House speaker's speech appeared to be slurred.
SEE: Managing AI and ML in the enterprise 2020: Tech leaders increase project development and implementation (TechRepublic Premium)
To create new content, GANs pit two neural networks against each other. The first one, called the generator, mimics the faces, landscapes, buildings and words it has seen and heard in the training dataset; and the second one, the discriminator, classifies the generator's work as either fake or real. The generator learns from this feedback and iteratively builds new media until the images and sounds produced are convincing enough to pass for real.
David Bau, a PhD student at MIT, told ZDNet: "We expose self-programmed systems to data, and then use an optimization procedure to let the algorithms decide for themselves how they'll do the computation. But humans don't decide how the programs are going to work. It's almost as if the algorithm comes up with some kind of magical formula."
To understand how GANs are wired, Bau began probing the deep nets and the millions of mathematical operations that underpin the algorithm's decisions. The scientist organized his GANs' neural programs as a record, to lay the algorithms down and visualize how the system worked, much like a developer would debug code.
After months of experiments, Bau realized that individual sets of neurons encoded for the presence of specific features in the content created by a GAN. More importantly, the scientist found that those neurons could be changed directly to edit the final content produced by the algorithm.
For example, you could search the neural network for the set of neurons that encode for the presence of a cloud in the sky, and by changing the set, add more clouds, or on the contrary erase the neurons to leave the sky blue and cloudless.
For the researcher, the discovery was an open door to a new GAN world, in which content would only be limited by human imagination. Think hats on horses, trees in the sky and other Studio Ghibli-esque fantastical creations.
But soon, Bau found that the algorithms wouldn't allow him to carry out certain modifications. Ask a GAN to draw a door in the sky, and it won't budge. "It will take that request and veto it," said Bau, "because that's very unrealistic."
The scientist set out to find out the obscure rules that GANs follow when they are creating new content. If an algorithm is tasked with generating a landscape, he asked himself, how does it know that clouds go in the sky, while doors are firmly reserved for buildings?
As it turns out, and after digging through the deeper layers of the neural network, Bau and his team discovered that GANs develop associative memory. Fed enough pictures of doors and clouds, the technology effectively learns the rules: doors for buildings, and clouds for skies.
But once the location of these conditional statements were located in the neural network, the researchers were able to manipulate the algorithm's memory to, literally, edit the way that the technology sees the world.
"I went in and saw there was a rule that seemed to be defining how kids' eyebrows are drawn," said Bau. "So I found where the computer remembered the fact that kids don't have furry eyebrows, and I rewrote the memory. And I got these very furried-eyebrowed kids out, and it was beautiful."
MIT's research team has now designed an intuitive interface to edit and rewrite memories for GANs, which lets users create new models that churn out content defined by human imagination.
MIT's new interface lets users rewrite GAN memory. In this case, the algorithm has forgotten that horses don't wear hats.
GAN editing has its limitations. The algorithms remain stubborn: Bau explained that when he tries to get a model to draw a door in the sky, for example, the feature is likely to look a lot like a cloud. This is because the technology's vision for the geometry of what typically goes in the sky is still influenced by what the GAN would normally see up there.
Nonetheless, Bau's discovery is a powerful insight into the way that the technology "thinks" – and, crucially, it reveals how strict GANs can be in copying the world as it exists today, including its flaws and biases.
SEE: AI-based traffic management gets green light
The scientist remembers a GAN model that used to include watermarks on a proportion of the images it generated, because the dataset included a small amount of copyrighted images. As harmless as the example is, it also reflects why exact copying might not be what researchers should expect from their algorithms.
"The GAN is making it much harder to create a world that is different to the one we know today," said Bau. "That concerns me because it feels very limiting. We don't want systems that do that."
Rather than letting the algorithms work their way in a black box, the researcher wants to increase the importance of human intent and imagination in technology. In fact, Bau hopes that artists will be the first users that leverage his new tool, to create pictures of a world that doesn't exist. "The real challenge I'm trying to breach here," he says, "is how to create models of the world based on people's imagination."