Appearance Manifold of Facial Expression

  • Caifeng Shan
  • Shaogang Gong
  • Peter W. McOwan
Conference paper
Part of the Lecture Notes in Computer Science book series (LNCS, volume 3766)


This paper investigates the appearance manifold of facial expression: embedding image sequences of facial expression from the high dimensional appearance feature space to a low dimensional manifold. We explore Locality Preserving Projections (LPP) to learn expression manifolds from two kinds of feature space: raw image data and Local Binary Patterns (LBP). For manifolds of different subjects, we propose a novel alignment algorithm to define a global coordinate space, and align them on one generalized manifold. Extensive experiments on 96 subjects from the Cohn-Kanade database illustrate the effectiveness of the alignment algorithm. The proposed generalized appearance manifold provides a unified framework for automatic facial expression analysis.


Facial Expression Face Image Local Binary Pattern Facial Expression Recognition Locally Linear Embedding 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. 1.
    Belkin, M., Niyogi, P.: Laplacian eigenmaps and spectral techniques for embedding and clustering. In: NIPS (2001)Google Scholar
  2. 2.
    Chang, Y., Hu, C., Turk, M.: Mainfold of facial expression. In: IEEE International Workshop on Analysis and Modeling of Faces and Gestures (2003)Google Scholar
  3. 3.
    Chang, Y., Hu, C., Turk, M.: Probabilistic expression analysis on manifolds. In: CVPR (2004)Google Scholar
  4. 4.
    Cohen, N., Sebe, A., Garg, L.: Facial expression recognition from video sequences: Temporal and static modeling. In: CVIU (2003)Google Scholar
  5. 5.
    Fasel, B., Luettin, J.: Automatic facial expression analysis: a survey. Pattern Recognition 36, 259–275 (2003)zbMATHCrossRefGoogle Scholar
  6. 6.
    Fidaleo, D., Trivedi, M.: Manifold analysis of facial gestures for face recognition. In: ACM SIGMM Multimedia Biometrics Methods and Application Workshop (2003)Google Scholar
  7. 7.
    He, X., Niyogi, P.: Locality preserving projections. In: NIPS (2003)Google Scholar
  8. 8.
    He, X., Yan, S., Hu, Y., Niyogi, P., Zhang, H.: Face recognition using laplacianfaces. IEEE PAMI 27(3), 328–340 (2005)Google Scholar
  9. 9.
    Hu, C., Chang, Y., Feris, R., Turk, M.: Manifold based analysis of facial expression. In: CVPR Workshop on Face Processing in Video (2004)Google Scholar
  10. 10.
    Kanade, T., Cohn, J.F., Tian, Y.: Comprehensive database for facial expression analysis. In: IEEE FG (2000)Google Scholar
  11. 11.
    Ojala, T., Pietikäinen, M., Mäenpää, T.: Multiresolution gray-scale and rotation invariant texture classification with local binary patterns. In: IEEE PAMI (2002)Google Scholar
  12. 12.
    Pantic, M., Rothkrantz, L.: Automatic analysis of facial expressions: the state of art. IEEE PAMI 22(12), 1424–1445 (2000)Google Scholar
  13. 13.
    Pantic, M., Rothkrantz, L.: Toward an affect-sensitive multimodal humancomputer interaction. Proceeding of the IEEE (2003)Google Scholar
  14. 14.
    Saul, L.K., Roweis, S.T.: Think globally, fit locally: Unsupervised learning of low dimensional manifolds. Journal of Machine Learning Research (2003)Google Scholar
  15. 15.
    Shan, C., Gong, S., McOwan, P.W.: Conditional Mutual Information Based Boosting for Facial Expression Recognition. In: BMVC (2005)Google Scholar
  16. 16.
    Shan, C., Gong, S., McOwan, P.W.: Robust facial expression recgnition using local binary patterns. In: IEEE ICIP (2005)Google Scholar
  17. 17.
    Tenenbaum, J.B., Silva, V., Langford, J.C.: A global geometric framework for nonlinear dimensionality reduction. Science 290 (2000)Google Scholar
  18. 18.
    Tian, Y.: Evaluation of face resolution for expression analysis. In: CVPR Workshop on Face Processing in Video (2004)Google Scholar
  19. 19.
    Tian, Y., Kanade, T., Cohn, J.: Recognizing action units for facial expression analysis. IEEE PAMI 23(2) (2001)Google Scholar
  20. 20.
    Tian, Y., Kanade, T., Cohn, J.: Facial Expression Analysis, Handbook of Face Recognition. Springer, Heidelberg (2003)Google Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2005

Authors and Affiliations

  • Caifeng Shan
    • 1
  • Shaogang Gong
    • 1
  • Peter W. McOwan
    • 1
  1. 1.Department of Computer ScienceQueen Mary, University of LondonLondonUK

Personalised recommendations