Face Morphing

Overview

  A morph is a simultaneous warp of the image shape and a cross-dissolve of the image colors. The cross-dissolve is the easy part; controlling and doing the warp is the hard part. The warp is controlled by defining a correspondence between the two pictures. The correspondence should map eyes to eyes, mouth to mouth, chin to chin, ears to ears, etc., to get the smoothest transformations possible.

  In this assignment I produced a "morph" animation of the faces of two different people, computed the mean of a population of faces and extrapolated from a population mean to create a caricature of myself.

Part 1: Defining Correspondences

  First, I manually defined pairs of corresponding points on the two target images using a consistent labeling, in order for the morph to work. I labeled the faces A and B in a consistent manner using the same ordering of keypoints in the two faces.

  For this part, I used the portraits of George and our old friend Derek! I selected 75 points in total across key facial features and accessories in both images by hand to ensure a smooth morphing process.

  I computed the Delaunay triangulation at a midway shape to lessen the potential triangle deformations. Here is the triangulation of these points.

Part 2: Computing the "Mid-way Face"

  Before computing the whole morph sequence, we need to compute the mid-way face. This would involve:

1. Computing the average shape (a.k.a the average of each keypoint location in the two faces).
2. Warping both faces into that shape.
3. Averaging the colors together.

  The main task in warping the faces into the average shape is implementing an affine warp for each triangle in the triangulation from the original images into this new shape. Instead of looping over the pixels, we iterated through all the triangles pairs. For each triangle in the average shape, we got the affine projection to the corresponding triangles in each image. Then, for each pixel in the triangle, we found the corresponding points in each image and set value to weighted average (cross-dissolve each triangle).

  To create the mid-way face, we warped both faces to the average shape and then blended them using equal weights, i.e. $0.5$. The original images and the image of the mid-way face are shown as below:

Part 3: The Morph Sequence

  Then we could generate a video sequence of the morph. I followed the method mentioned in the course.

1. Define corresponding points.
2. Define triangle on points.
   • Use same triangulation for both images.
3. For each $t = 0 :$ step $: 1$
   • Compute the average shape at $t$ (weighted average of points).
   • For each triangle in the average shape:
     • Get the affine projection to the correponding triangles in each image.
     • For each pixel in the triangle, find the correponding points in each image and set value to weighted average (cross-dissolve each triangle).
   • Save the image as the nect frame of the sequence.

George

Morphing Video

Derek

Part 4: The "Mean face" of a population

  I analyzed a free data set of Danish faces for this part. There are 40 people in this dataset.

  I computed the average face shape of the whole population and some subsets of the population - say, all the men & women.

  Then I morphed each of the faces in the dataset into the average shape, i.e. the mean Danish face.

  Below show my face warped into the average geometry and the mean Danish face warped into my geometry.

Part 5: Caricatures: Extrapolating from the mean

  I also produced a caricature of my face by extrapolating from the population mean I calculated in the last step, i.e. mean Danish face. This procedure follows: $$ \text{caricature} = \text{average face} + \alpha \times (\text{my face} - \text{average face}). $$

  It is obvious that the further $\alpha$ deviates from $1.0$, the stranger my face looks. That's because the caricature is the same as my face when $\alpha = 1.0$. You could deduce this from the equation above.

Bells and Whistles

Change age/gender/ethnicity/smile of my face.

  I changed my face into a smily Danish man!

Morph the Shape

Morph the Appearance

Complete Morph

Make a morphing music video on a theme.

  I made a morphing music video of the main characters in EVANGELION. Here I provide a gif version for you, you could find the mp4 version with music Here! (The quality of the gif file might be poor due to the compression operation, so please click on the link to view the original video)