Lecture 19: Introduction to Character Animations (23)

philippe-eecs

This blending of faces reminds me of computer vision in deep learning where one uses a VAE to learn a latent space representing all faces. Then takes the portion of the latent space related to anger, a portion of the latent space related to sadness, and adds them together to produce a point in the latent space that represents betrayal. When you decode from this latent space the face is truly a mix-up.

ronthalanki

There is also a more classical way of doing what @philippe-eecs mentioned. You should be able to use a method used for computational caricatures. Here is a lecture from cs194-26 that explains how caricatures are computed: https://inst.eecs.berkeley.edu/~cs194-26/fa20/Lectures/faces.pdf

Basically, you take an average of all faces for a particular emotion and then calculate the difference between a particular face and the average face. Then you add the vector between the average face and the average face for a particular emotion.

ada-hu

I thought this slide was particularly interesting, not just from a technical viewpoint, but also from a human one - I thought it was super cool how we could get other emotions by combining the key shapes and expressions of two "base" expressions

This blending of faces reminds me of computer vision in deep learning where one uses a VAE to learn a latent space representing all faces. Then takes the portion of the latent space related to anger, a portion of the latent space related to sadness, and adds them together to produce a point in the latent space that represents betrayal. When you decode from this latent space the face is truly a mix-up.

There is also a more classical way of doing what @philippe-eecs mentioned. You should be able to use a method used for computational caricatures. Here is a lecture from cs194-26 that explains how caricatures are computed: https://inst.eecs.berkeley.edu/~cs194-26/fa20/Lectures/faces.pdf

Basically, you take an average of all faces for a particular emotion and then calculate the difference between a particular face and the average face. Then you add the vector between the average face and the average face for a particular emotion.

I thought this slide was particularly interesting, not just from a technical viewpoint, but also from a human one - I thought it was super cool how we could get other emotions by combining the key shapes and expressions of two "base" expressions