How Microsoft's caricature AI turns photos into portrait cartoons
To become an outstanding cartoonist can take years of practice, but that skill -- or at least the facial caricature part -- has now been automated by a group of Microsoft researchers.
AR + VR
The group's work employs a type of AI algorithm called generative adversarial networks (GAN), the favored deep-learning technique for creating 'deep fakes' or manipulating real video and images to create very convincing fakes.
Instead of deep fakes, Kaidi Cao of Tsinghua University, Jing Liao of Microsoft Research, and Lu Yuan from Microsoft AI Perception and Mixed Reality, used GANs to create a cartoon generator capable of outputting caricatures like a human artist.
The deep neural network consists of two caricature GANs, or CariGANs: CariGeoGAN for modeling the geometric exaggeration of a person's face and transferring it from photo to cartoon, while CariStyGAN transfers the style of caricatures to a given face picture.
The researchers focus on shape exaggeration and appearance stylization on the basis that these are the two key aspects to drawing a caricature that is warped but still recognizable.
As a cartoon-generating app, the researchers configured the CariGANs so that a user can control how exaggerated the face becomes and the style of the caricature by tuning the parameters of the model or by supplying an example caricature that the GAN can mimic.
To capture various caricature styles and how features are exaggerated, the researchers drew on over 8,000 caricatures found on the internet. The photos were drawn from Microsoft's MS-Celeb-1M dataset, previously used in an AI project to recognize celebrities.
And to test how recognizable the CariGAN caricatures are, the researchers conducted two perceptual tests, showing human participants a selection of its caricatures and asking them to pick the one correct subject from a choice of five photos of faces with similar attributes.
SEE: How to implement AI and machine learning (ZDNet special report) | Download the report as a PDF (TechRepublic)
The study found that hand-drawn characters had the highest recognition rate among participants. However, CariGAN performed better on this measure than previous techniques for generating characters.
The second study tested how faithful CariGAN outputs were to the hand-drawn caricature styles. To do this they showed participants eight caricatures by artists showing a particular style.
Then they showed one hand-drawn image of a subject and five caricatures from the CariGAN model of the same subject. Next, they asked participants to rank them from "the most similar to given caricature samples" to "the least similar to caricature".
Participants ranked the CariGAN output better than the hand-drawn one 22.95 percent of the time, meaning the AI output is indistinguishable from real hand-drawn caricatures sometimes. The researchers note an "ideal fooling rate" would be 50 percent of the time.
Anyone interested in taking the tests the study's participants did can try them in the supplementary materials link on the CariGAN project page.
The photos were drawn from Microsoft's MS-Celeb-1M dataset, previously used in an AI project to recognize celebrities.
Previous and related coverage
Making sense of Microsoft's approach to AI
Microsoft has a master plan for trying to get more customers to jump into the AI waters. Here's an attempt to explain how containers, accelerators, and APIs all figure in.
Windows 10: How Microsoft used AI to speed up its April 2018 Update rollout
Microsoft said it used AI technologies to monitor its latest Windows 10 update.
Microsoft Translator: Now AI works offline for Android, iOS, with Windows due soon
New AI-powered offline translation packs are available for end-users and developers.
Microsoft buys into Grab as pair focus on big data and AI on Azure
Grab will adopt Microsoft Azure as its preferred cloud platform and Microsoft will make a strategic investment in Grab.
Ignite 2018: Microsoft steps up data platform and AI ambitions
Microsoft unveils big-data-capable SQL Server 2019 and extended AI capabilities to power data-driven innovation.
Five steps for getting started with AI in your business TechRepublic
A new report by Microsoft identifies how businesses are getting to grip with technologies such as machine learning.
Microsoft to acquire chatbot startup as it pushes for 'responsible AI' CNET
XOXCO is the creator of Slack bot Howdy and Botkit on GitHub.