A Study of Detecting Social Interaction with Sensors in a Nursing Home Environment

  • Datong Chen
  • Jie Yang
  • Howard Wactlar
Conference paper
Part of the Lecture Notes in Computer Science book series (LNCS, volume 3766)


Social interaction plays an important role in our daily lives. It is one of the most important indicators of physical or mental diseases of aging patients. In this paper, we present a Wizard of Oz study on the feasibility of detecting social interaction with sensors in skilled nursing facilities. Our study explores statistical models that can be constructed to monitor and analyze social interactions among aging patients and nurses. We are also interested in identifying sensors that might be most useful in interaction detection; and determining how robustly the detection can be performed with noisy sensors. We simulate a wide range of plausible sensors using human labeling of audio and visual data. Based on these simulated sensors, we build statistical models for both individual sensors and combinations of multiple sensors using various machine learning methods. Comparison experiments are conducted to demonstrate the effectiveness and robustness of the sensors and statistical models for detecting interactions.


Nursing Home Information Gain Support Vector Machine Model Hand Gesture Sensor Output 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. 1.
    Aggarwal, J.K., Cai, Q.: Human Motion Analysis: A Review. Computer Vision and Image Understanding 73, 428–440 (1999)CrossRefGoogle Scholar
  2. 2.
    Brank, J., Grobelnik, M., Milic-Frayling, N., Mladenic, D.: Feature selection using linear support vector machines. MSR-TR-2002-63, Microsoft research (2002)Google Scholar
  3. 3.
    Bregler, C.: Learning and Recognizing Human Dynamics in Video Sequences. In: CVPR, pp. 568–574 (1997)Google Scholar
  4. 4.
    Brumitt, B., Krumm, J., Meyers, B., Shafer, S.: Ubiquitous computing and the role of geometry. In: Special Issue on Smart Spaces and Environments. IEEE Personal Communications, vol. 7-5, pp. 41–43 (October 2000)Google Scholar
  5. 5.
    Carp, F.: Assessing the environment. Annul review of gerontology and geriatrics 14, 302–314 (1994)Google Scholar
  6. 6.
    Clarkson, B., Pentland, A.: Framing Through Peripheral Perception. In: Proc. of ICIP, Vancouver (September 2000)Google Scholar
  7. 7.
    Clarkson, B., Pentland, A.: Unsupervised Clustering of Ambulatory Audio and Video. In: Proc. of the ICASSP, Phoenix (1998)Google Scholar
  8. 8.
    Emler, N.: Gossip, reputation, and social adaptation. In: Goodman, R.F., Ben-Ze’ev, A. (eds.) Good Gossip, pp. 117–138. University Press of Kansas, Wichita (1994)Google Scholar
  9. 9.
    Eppig, F.J., Poisal, J.A.: Mental health of medicare beneficiaries: 1995. Health Care Financing Review 15, 207–210 (1995)Google Scholar
  10. 10.
    Essa, I., Pentland, A.: Facial expression recognition using a dynamic model and motion energy. In: Proc. 5th Intl. Conf. on Computer Vision, pp. 360–367 (1995)Google Scholar
  11. 11.
    Freeman, W.T., Roth, M.: Orientation histograms for hand gesture recognition. In: International Workshop on Automatic Face and Gesture Recognition, pp. 296–301 (June 1995)Google Scholar
  12. 12.
    Friedman, J., Hastie, T., Tibshirani, R.: Additive logistic regression: a statistical view of boosting. Annals of Statistics 28, 307–337 (2000)CrossRefMathSciNetGoogle Scholar
  13. 13.
    German, P.S., Rovner, B.W., Burton, L.C., Brant, L.J., Clark, R.: The role of mental morbidity in the nursing home experience. Gerontologist 32(2), 152–158 (1992)Google Scholar
  14. 14.
    Hastie, T., Tibshirani, R.: Nonpararmetric logistic and proportional odds regression. Applied statistics 36, 260–276 (1987)CrossRefGoogle Scholar
  15. 15.
    Harter, A., Hopper, A., Steggles, P., Ward, A., Webster, P.: The anatomy of a contextaware application. In: Proceedings of the 5th Annual ACM/IEEE International Conference on Mobile Computing and Networking, Seattle, WA, pp. 59–68 (August 1999)Google Scholar
  16. 16.
    Holmquist, L., Falk, J., Wigstrm, J.: Supporting group collaboration with interpersonal awareness devices. Personal Technologies 3, 13–21 (1999)CrossRefGoogle Scholar
  17. 17.
    Hooyman, N.R., Kiyak, H.: Social Gerontology: A Multidisciplinary Perspective, 6th edn. Allyn and Bacon (2002)Google Scholar
  18. 18.
    Hudson, S., Fogarty, J., Atkeson, C., Avrahami, D., Forlizzi, J., Kiesler, S., Lee, J., Yang, J.: Predicting Human Interruptibility with Sensors: A Wizard of Oz Feasibility Study. In: Proc. of the SIGCHI Conference on Human Factors in Computing Systems, pp. 257–264 (2003)Google Scholar
  19. 19.
    Jug, M., Pers, J., Dezman, B., Kovacic, S.: Trajectory based assessment of coordinated human activity. In: ICVS 2003, pp. 534–543 (2003)Google Scholar
  20. 20.
    Kidd, C.D., Orr, R., Abowd, G.D., Atkeson, C.G., Essa, I.A., Macintyre, B., Mynatt, E., Starner, T.E., Newstetter, W.: The Aware Home: A Living Laboratory for Ubiquitous Computing Research. In: Streitz, N.A., Hartkopf, V. (eds.) CoBuild 1999. LNCS, vol. 1670, pp. 191–198. Springer, Heidelberg (1999)CrossRefGoogle Scholar
  21. 21.
    Koile, K., Tollmar, K., Demirdjian, D., Shrobe, H.E., Darrell, T.: Activity Zones for Context- Aware Computing. In: Dey, A.K., Schmidt, A., McCarthy, J.F. (eds.) UbiComp 2003. LNCS, vol. 2864, pp. 90–106. Springer, Heidelberg (2003)CrossRefGoogle Scholar
  22. 22.
    Kononenko, I.: Semi-naive bayesian classifier. In: Proceedings of sixth European Working Session on Learning, pp. 206–219. Springer, Heidelberg (1991)Google Scholar
  23. 23.
    Lee, S., Mase, K.: Activity and location recognition using wearable sensors. In: 1st IEEE International Conference on Pervasive Computing and Communications, pp. 24–32 (2002)Google Scholar
  24. 24.
    Lubinski, R.: Dementia and communication. In: B. C, Decker, Philadelphia (1991)Google Scholar
  25. 25.
    Margineantu, D.D., Dietterich, T.G.: Pruning adaptive boosting. In: 14th Int. Conf. on Machine Learning, pp. 211–218. Morgan Kaufmann, San Francisco (1997)Google Scholar
  26. 26.
    Martin, A., Karrray, L., Gilloire, A.: High Order Statistics for Robust Speech/Non- Speech Detection. In: Eusipco, Tampere, Finland, pp. 469–472 (September 2000)Google Scholar
  27. 27.
    Nelson, J.: The influence of environmental factors in incidents of disruptive behavior. Journal of Gerontological Nursing 21(5), 19–24 (1995)Google Scholar
  28. 28.
    Quinlan, J.R.: C4.5: programs for machine learning. Morgan Kaufmann, San Francisco (1993)Google Scholar
  29. 29.
    Rhodes, B.: The wearable remembrance agent: A system for augmented memory. In: Proceedings of the 1st International Symposium on Wearable Computers, pp. 123–128 (1997)Google Scholar
  30. 30.
    Schraudolph, N., Sejnowski, T.J.: Unsupervised discrimination of clustered data via optimization of binary information gain. In: Hanson, S.J., Cowan, J.D., Lee Giles, C. (eds.) Advances in Neural Information Processing Systems, vol. 5, pp. 499–506. Morgan Kaufmann, San Mateo (1993)Google Scholar
  31. 31.
    Sloane, P.D., Mitchell, C.M., Long, K., Lynn, M.: TESS 2+ Instrument B: Unit observation checklist – physical environment: A report on the psychometric properties of individual items, and initial recommendations on scaling. University of North Carolina (1995)Google Scholar
  32. 32.
    Steele, C., Rovner, B.W., Chase, G.A., Folstein, M.: Psychiatric symptoms and nursing home placement in Alzheimer’s disease. American Journal of Psychiatry 147(8), 1049–1051 (1990)Google Scholar
  33. 33.
    Stauffer, C., Grimson, W.E.L.: Adaptive background mixture models for real-time tracking. In: Proc. of CVPR 1999 (1999)Google Scholar
  34. 34.
    Time Domain Corporation, 7057 Old Madison Pike, Huntsville, AL 35806. PulsON Technology: Time Modulated Ultra Wideband Overview (2001) Google Scholar
  35. 35.
    Vapnik, V.N.: The nature of statistical learning theory. Springer, New York (1995)zbMATHGoogle Scholar
  36. 36.
    Zhang, D., Li, S.Z., Gatica-Perez, D.: Real-Time Face Detection Using Boosting Learning in Hierarchical Feature Spaces. In: 17th International Conference on Pattern Recognition (2004)Google Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2005

Authors and Affiliations

  • Datong Chen
    • 1
  • Jie Yang
    • 1
  • Howard Wactlar
    • 1
  1. 1.School of Computer ScienceCarnegie Mellon University 

Personalised recommendations