Attention Monitoring Based on Temporal Signal-Behavior Structures

  • Akira Utsumi
  • Shinjiro Kawato
  • Shinji Abe
Conference paper
Part of the Lecture Notes in Computer Science book series (LNCS, volume 3766)


In this paper, we discuss our system that estimates user attention to displayed content signals with temporal analysis of their exhibited behavior. Detecting user attention and controlling contents are key issues in our “networked interaction therapy system,” which effectively attracts the attention of memory-impaired people. In our proposed system, user behavior, including facial movements and body motions (“beat actions”), is detected with vision-based methods. User attention to the displayed content is then estimated based on the on/off facial orientation from a display system and body motions synchronous to auditorial signals. This attention monitoring mechanism design is derived from observations of actual patients. Estimated attention level can be used for content control to attract more attention of the viewers to the display system. Experimental results suggest that the content switching mechanism effectively attracts user interest.


Video Content Gesture Recognition Content Control Facial Orientation Face Tracking 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. 1.
    Kuwahara, N., Kuwabara, K., Utsumi, A., Yasuda, K., Tetsutani, N.: Networked interaction therapy: Relieving stress in memory-impaired people and their family members. In: Proc. of IEEE Engineering in Medicine and Biology Society (2004)Google Scholar
  2. 2.
    Utsumi, A., Ohya, J.: Multiple-camera-based human tracking using nonsynchronous observations. In: Proceedings of Fourth Asian Conference on Computer Vision, pp. 1034–1039 (2000)Google Scholar
  3. 3.
    Gavrila, D.M., Davis, L.S.: 3-d model-based tracking of humans in action: a multi-view approach. In: Proc. of Computer Vision and Pattern Recognition, pp. 73–80 (1996)Google Scholar
  4. 4.
    Cai, Q., Aggarwal, J.K.: Tracking human motion using multiple cameras. In: Proceedings of 13th International Conference on Pattern Recognition, pp. 68–72 (1996)Google Scholar
  5. 5.
    Segen, J., Pingali, S.: A camera-based system for tracking people in real time. In: Proceedings of 13th International Conference on Pattern Recognition, pp. 63–67 (1996)Google Scholar
  6. 6.
    Low, J., Schietecat, T., Kwok, T.F., Lindeboom, L.: Technology applied to address difficulties of alzheimer patients and their partners. In: Proceedings of the conference on Dutch directions in HCI, p. 18. ACM Press, New York (2004)Google Scholar
  7. 7.
  8. 8.
    Yang, J., Stiefelhagen, R., Meier, U., Waibel, A.: A real-time face tracker. In: Proc. 3rd IEEE Workshop on Application of Computer Vision, pp. 142–147 (1996)Google Scholar
  9. 9.
    Terrillon, J.C., David, M., Akamatsu, S.: Automatic detection of human faces in natural scene image by use of skin color model and invariant moment. In: Proc. of IEEE Int. Conf. on Automatic Face and Gesture Recognition, pp. 112–117 (1998)Google Scholar
  10. 10.
    Heinzmann, J., Zelinsky, A.: 3-d facial pose and gaze point estimation using a robust real time tracking paradigm. In: Proc. of IEEE Int. Conf. on Automatic Face and Gesture Recognition, pp. 142–147 (1998)Google Scholar
  11. 11.
    Kawato, S., Tetsutani, N.: Real-time detection of between-the-eyes with a circle frequency filter. In: Proc. of ACCV 2002, pp. 442–447 (2002)Google Scholar
  12. 12.
    Gorodnichy, D.O.: On importance of nose for face tracking. In: Proc. of IEEE 5th Int. Conf. on Automatic Face and Gesture Recognition, pp. 188–193 (2002)Google Scholar
  13. 13.
    Scheirer, E.D.: Tempo and beat analysis of acoustic musical signals. Journal of Accoustical Society of America 103 (1), 588–601 (1998)CrossRefGoogle Scholar
  14. 14.
    Goto, M.: An audio-based real-time beat tracking system for music with or without drum-sounds. Journal of New Music Research 30(2), 159–171 (2001)CrossRefGoogle Scholar
  15. 15.
    Shiratori, T., Nakazawa, A., Ikeuchi, K.: Detecting dance motion structure through music analysis. In: Proc. of Sixth International Conference on Automatic Face and Gesture Recognition, pp. 857–862 (2004)Google Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2005

Authors and Affiliations

  • Akira Utsumi
    • 1
  • Shinjiro Kawato
    • 1
  • Shinji Abe
    • 1
  1. 1.ATR Intelligent Robotics and Communication LaboratoriesKyotoJapan

Personalised recommendations