An Objective Character Believability Evaluation Procedure for Multi-agent Story Generation Systems

  • Mark O. Riedl
  • R. Michael Young
Conference paper
Part of the Lecture Notes in Computer Science book series (LNCS, volume 3661)


The ability to generate narrative is of importance to computer systems that wish to use story effectively for entertainment, training, or education. One of the focuses of intelligent virtual agent research in general and story generation research in particular is how to make agents/characters more lifelike and compelling. However, one question that invariably comes up is: Is the generated story good? An easier question to tackle is whether a reader/viewer of a generated story perceives certain essential attributes such as causal coherence and character believability. Character believability is the perception that story world characters are acting according to their own beliefs, desires, and intentions. We present a novel procedure for objectively evaluating stories generated for multiple agents/characters with regard to character intentionality – an important aspect of character believability. The process transforms generated stories into a standardized model of story comprehension and then indirectly compares that representation to reader/viewer mental perceptions about the story. The procedure is illustrated by evaluating a narrative planning system, Fabulist.


Character Action Character Believability Story Comprehension Character Intentionality Quest Model 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. 1.
    Meehan, J.R.: The Metanovel: Writing Stories by Computer. Ph.D. Dissertation, Yale University (1976)Google Scholar
  2. 2.
    Cavazza, M., Charles, F., Mead, S.: Planning characters’ behaviour in interactive storytelling. Journal of Visualization and Computer Animation 13, 121–131 (2002)zbMATHCrossRefGoogle Scholar
  3. 3.
    Charles, F., Cavazza, M.: Exploring the scalability of character-based storytelling. In: Proceedings of the 3rd International Joint Conference on Autonomous Agents and Multi Agent Systems (2004)Google Scholar
  4. 4.
    Lebowitz, M.: Story-telling as planning and learning. Poetics 14, 483–502 (1985)CrossRefGoogle Scholar
  5. 5.
    Riedl, M.O., Young, R.M.: An intent-driven planner for multi-agent story generation. In: Proceedings of the 3rd International Joint Conference on Autonomous Agents and Multi-Agent Systems (2003)Google Scholar
  6. 6.
    Riedl, M.O.: Narrative Planning: Balancing Plot and Character. Ph.D. Dissertation. North Carolina State University (2004)Google Scholar
  7. 7.
    Turner, S.R.: The Creative Process: A Computer Model of Storytelling, Hillsdale, NJ. Lawrence Erlbaum Associates, Mahwah (1994)Google Scholar
  8. 8.
    Szilas, N.: IDtension: A narrative engine for interactive drama. In: Proceedings of the 1st International Conference on Technologies for Interactive Digital Storytelling and Entertainment (2003)Google Scholar
  9. 9.
    Mateas, M.: Interactive Art, Drama, and Artificial Intelligence. Ph.D. Disssertation, Carnegie Mellon University (2002)Google Scholar
  10. 10.
    Mateas, M., Stern, A.: Integrating plot, character, and natural language processing in the interactive drama Façade. In: Proceedings of the 1st International Conference on Technologies for Interactive Digital Storytelling and Entertainment (2003)Google Scholar
  11. 11.
    Bates, J.: The role of emotion in believable agents. Communications of the ACM 37 (1994)Google Scholar
  12. 12.
    Sengers, P.: Narrative and schizophrenia in artificial agents. In: Mateas, M., Sengers, P. (eds.) Narrative Intelligence. John Benjamins, Amsterdam (2003)Google Scholar
  13. 13.
    Gerrig, R.J.: Experiencing Narrative Worlds: On the Psychological Activities of Reading. Yale University Press, New Haven (1993)Google Scholar
  14. 14.
    Graesser, A.C., Lang, K.L., Roberts, R.M.: Question answering in the context of stories. Journal of Experimental Psychology: General 120 (1991)Google Scholar
  15. 15.
    Christian, D.B., Young, R.M.: Comparing cognitive and computational models of narrative structure. In: Proceedings of the 19th National Conference on Artificial Intelligence (2004)Google Scholar
  16. 16.
    Young, R.M.: Notes on the use of planning structures in the creation of interactive plot. In: Mateas, M., Sengers, P. (eds.) Narrative Intelligence: Papers from the 1999 Fall Symposium, Menlo Park CA. American Association for Artificial Intelligence (1999)Google Scholar
  17. 17.
    Penberthy, J.S., Weld, D.: UCPOP: A sound, complete, partial-order planner for ADL. In: Proceedings of the 3rd International Conference on Knowledge Representation and Reasoning (1992)Google Scholar
  18. 18.
    Young, R.M., Moore, J.D., Pollack, M.E.: Towards a principled representation of discourse plans. In: Proceedings of the 16th Conference of the Cognitive Science Society (1994)Google Scholar
  19. 19.
    Callaway, C.B., Lester, J.C.: Narrative prose generation. Artificial Intelligence 139 (2002)Google Scholar
  20. 20.
    Sadock, J.M.: Comments on Vanderveken and on Cohen and Levesque. In: Cohen, P.R., Morgan, J., Pollack, M.E. (eds.) Intentions in Communication, pp. 257–270. The MIT Press, Cambridge (1990)Google Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2005

Authors and Affiliations

  • Mark O. Riedl
    • 1
  • R. Michael Young
    • 2
  1. 1.Institute for Creative TechnologiesUniversity of Southern CaliforniaLos AngelesUSA
  2. 2.Department of Computer ScienceNorth Carolina State UniversityRaleighUSA

Personalised recommendations