Where Do They Look? Gaze Behaviors of Multiple Users Interacting with an Embodied Conversational Agent

  • Matthias Rehm
  • Elisabeth André
Conference paper
Part of the Lecture Notes in Computer Science book series (LNCS, volume 3661)


In this paper, we describe an experiment we conducted to determine the user’s level of engagement in a multi-party scenario consisting of human and synthetic interlocutors. In particular, we were interested in the question of whether humans accept a synthetic agent as a genuine conversational partner that is worthy of being attended to in the same way as the human interlocutors. We concentrated on gaze behaviors as one of the most important predictors of conversational attention. Surprisingly, humans paid more attention to an agent that talked to them than to a human conversational partner. No such effect was observed in the reciprocal case, namely when humans addressed an agent as opposed to a human interlocutor.


Attentive Behavior Conversational Agent Human Player Conversational Partner Synthetic Agent 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. 1.
    Argyle, M., Cook, M.: Gaze and Mutual Gaze. Cambridge University Press, Cambridge (1976)Google Scholar
  2. 2.
    Bailenson, J.N., Blasovich, J., Beall, A.C., Loomis, J.M.: Interpersonal distance in immersive virtual environments. Personality and Social Psychology Bulletin 29(7), 819–833 (2003)CrossRefGoogle Scholar
  3. 3.
    Bergmann, J.R.: Haustiere als kommunikative Ressourcen. Soziale Welt: Zeitschrift für sozialwissenschaftliche Forschung und Praxis, Sonderband: Kultur und Alltag 8, 299–312 (1988)Google Scholar
  4. 4.
    BLAG. Berliner Lexikon der Alltagsgesten, (last visited: 09.12.2004)
  5. 5.
    Colburn, A., Cohen, M., Drucker, S.: The role of eye gaze in avatar mediated conversational interfaces (2000)Google Scholar
  6. 6.
    Ekman, P.: Telling Lies — Clues to Deceit in the Marketplace, Politics, and Marriage, 3rd edn. Norton and Co. Ltd., New York (1992)Google Scholar
  7. 7.
    Garau, M., Slater, M., Bee, S., Sasse, M.A.: The impact of eye gaze on communication using humanoid avatars. In: CHI 2001: Proceedings of the SIGCHI conference on Human factors in computing systems, pp. 309–316. ACM Press, New York (2001)CrossRefGoogle Scholar
  8. 8.
    Hartmann, B., Mancini, M., Pelachaud, C.: Formational parameters and adaptive prototype instantiation for mpeg-4 compliant gesture synthesis. In: CASA 2002, pp. 111–119 (2002)Google Scholar
  9. 9.
    Kendon, A.: Some functions of gaze direction in social interaction. Acta Psychologica 32, 1–25 (1967)Google Scholar
  10. 10.
    Kipp, M.: Anvil - a generic annotation tool for multimodal dialogue. In: Proceedings of the 7th European Conference on Speech Communication and Technology (Eurospeech), Aalborg, September 2001, pp. 1367–1370 (2001)Google Scholar
  11. 11.
    Lee, S.P., Badler, J.B., Badler, N.I.: Eyes alive. In: SIGGRAPH 2002: Proceedings of the 29th annual conference on Computer graphics and interactive techniques, pp. 637–644. ACM Press, New York (2002)CrossRefGoogle Scholar
  12. 12.
    Nakano, Y.I., Reinstein, G., Stocky, T., Cassell, J.: Towards a model of face-to-face grounding. In: Proceedings of the Annual Meeting of the Association for Computational Linguistics (ACL 2003), pp. 553–561 (2003)Google Scholar
  13. 13.
    Pelachaud, C., Poggi, I.: Subtleties of facial expressions in embodied agents. Journal of Visualization and Computer Animation 31, 301–312 (2002)CrossRefGoogle Scholar
  14. 14.
    Rehm, M., André, E.: Catch me if you can: Exploring lying agents in social settings. In: Proceedings of Conference on Autonomous Agents and Multi-Agent Systems (AAMAS). ACM Press, New York (2005)Google Scholar
  15. 15.
    Sidner, C.L., Kidd, C.D., Lee, C., Lesh, N.: Where to look: a study of human-robot engagement. In: IUI 2004: Proceedings of the 9th international conference on Intelligent user interface, pp. 78–84. ACM Press, New York (2004)CrossRefGoogle Scholar
  16. 16.
    Vertegaal, R., Slagter, R., Nijholt, A., van der Veer, G., Nijholt, A.: Eye gaze patterns in conversations: there is more to conversational agents than meets the eyes. In: CHI 2001: Proceedings of the SIGCHI conference on Human factors in computing systems, pp. 301–308. ACM Press, New York (2001)CrossRefGoogle Scholar
  17. 17.
    Vinayagamoorthy, V., Garau, M., Steed, A., Slater, M.: An eye gaze model for dyadic interaction in an immersive virtual environment: Practice and experience. Comput. Graph. Forum 23(1), 1–12 (2004)CrossRefGoogle Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2005

Authors and Affiliations

  • Matthias Rehm
    • 1
  • Elisabeth André
    • 1
  1. 1.Multimedia Concepts and ApplicationsUniversity of AugsburgGermany

Personalised recommendations