Datong Chen
ABSTRACT
In this paper, we propose an ontology-based approach for analyzing social interaction patterns in a nursing home from video. Social interaction patterns are broken into individual activities and behavior events using a multi-level context hierarchy ontology framework. To take advantage of an ontology in representing how social interactions evolve, we design and refine the ontology based on knowledge gained from 80 hours of video recorded in the public spaces of a nursing home. The ontology is implemented using a dynamic Bayesian network to statistically model the multi-level concepts defined in the ontology. We have developed a prototype system to illustrate the proposed concept. Experiment results have demonstrated feasibility of the proposed approach. The objective of this research is to automatically create concise and comprehensive reports of activities and behaviors of patients to support physicians and caregivers in a nursing facility
- J. K. Aggarwal, Q. Cai, "Human Motion Analysis: A Review," Computer Vision and Image Understanding, Vol. 73, pp. 428--440, 1999. Google Scholar
Digital Library
- K. Ahrens, S. F. Chung, and C. Huang, "Conceptual Metaphors: Ontology-based Representation and Corpora Driven Mapping Principles". In Proceedings of the ACL Workshop on the Lexicon and Figurative Language. Google ScholarDigital Library
- D. Ayers, M. Shah, "Monitoring Human Behavior from Video Taken in an Office Environment," Image and Vision Computing, Vol. 19, pp. 833--846, 2001.Google ScholarCross Ref
- N. Badler, "Temporal Scene Analysis: Conceptual Description of Object Movements," University of Toronto Technical Report No. 80, 1975.Google Scholar
- B. Brumitt, J. Krumm, B. Meyers, and S. Shafer, "Ubiquitous computing and the role of geometry". In Special Issue on Smart Spaces and Environments, volume 7-5, pages 41--43. IEEE Personal Communications, October 2000.Google ScholarCross Ref
- F. Carp, "Assessing the environment", Annul review of gerontology and geriatrics, 14, pages: 302--314, 1994.Google Scholar
- J. W. Davis, A. F. Bobick, "The Representation and Recognition of Human Movement Using Temporal Templates," Proc. of CVPR, pp. 928--934, 1997. Google ScholarDigital Library
- A. Doucet, N. Freitas, and N. Gordon. Sequential Monte Carlo Methods in Practice. Springer-Verlag, 2001.Google ScholarCross Ref
- F. J. Eppig and J. A. Poisal, "Mental health of medicare beneficiaries: 1995", Health Care Financing Review, 15, pages: 207--210, 1995.Google Scholar
- S. Farrar, W. Lewis, and T. Langendoen, "A Common Ontology for Linguistic Concepts". In Proceedings of the Knowledge Technologies Conference, 2002.Google Scholar
- H. Hartley, "Maximum likelihood estimation from incomplete data". Bio-metrics, 14:174--194, 1958.Google Scholar
- A. Harter, A. Hopper, P. Steggles, A. Ward, and P. Webster, "The anatomy of a context-aware application". In Proceedings of the 5th Annual ACM/IEEE International Conference on Mobile Computing and Networking, pages 59--68, Seattle, WA, August 1999. ACM Press. Google ScholarDigital Library
- S. Hongeng, R. Nevatia, "Multi-Agent Event Recognition," International Conference on Computer Vision, pp. 84--91, 2001.Google Scholar
- S. Intille and A. Bobick, "Recognizing planned, multi-person action", Computer Vision and Image Understanding, vol. 81, no. 3, pp. 414--445, March 2001 Google ScholarDigital Library
- Y. A. Ivanov, A. F. Bobick, "Recognition of Visual Activities and Interactions by Stochastic Parsing," IEEE Transactions of Pattern Analysis and Machine Intelligence, Vol. 22, pp. 852--872, 2000. Google ScholarDigital Library
- T. Jebara, A. Pentland, "Action Reaction Learning: Analysis and Synthesis of Human Behavior," IEEE Workshop on the Interpretation of Visual Motion, 1998.Google Scholar
- C. D. Kidd, R. Orr, G. D. Abowd, C. G. Atkeson, I. A. Essa, B. Macintyre, E. Mynatt, and T. E. Starner and W. Newstetter, "The Aware Home: A Living Laboratory for Ubiquitous Computing Research". Proc. of CoBuild '99, pp.191--198, 1999. Google ScholarDigital Library
- K. Koile, K. Tollmar, D. Demirdjian, H. E. Shrobe, T. Darrell, "Activity Zones for Context-Aware Computing", Ubicomp 2003, pp. 90--106, 2003.Google Scholar
- A. Kojima, T. Tamura, "Natural Language Description of Human Activities from Video Images Based on Concept Hierarchy Actions," International Journal of Computer Vision, Vol. 50, pp. 171--184, 2001. Google ScholarDigital Library
- R. Lubinski, "Dementia and communication", Philadelphia: B. C. Decker, 1991.Google Scholar
- I. McCowan, D. Gatica-Perez, S. Bengio, G. Lathoud, M. Barnard and Dong Zhang, "Automatic Analysis of Multi-modal Group Actions in Meetings," IEEE Transactions on PAMI, 2004. Google ScholarDigital Library
- D. J. Moore, I. A. Essa, M. H. Hayes, "Exploiting Human Actions and Object Context for Recognition Tasks," Proc. of ICCV, Vol. 1, pp. 80--86, 1999.Google Scholar
- J. Nelson, "The influence of environmental factors in incidents of disruptive behavior", Journal of Gerontological Nursing 21(5):19--24, 1995.Google ScholarCross Ref
- I. Niles and A. Pease, "Towards a Standard Upper Ontology". In Proceedings of the 2nd International Conference on Formal Ontology in Information Systems (FOIS-2001), Chris Welty and Barry Smith, eds, 2001. Google ScholarDigital Library
- K. Nummiaro, E. Koller-Meier, and L. Van Gool. Object tracking with an adaptive color-based particle filter. In Proc. Symposium for Pattern Recognition of the DAGM, Sep. 2000. Google ScholarDigital Library
- N. Oliver, A. Garg, E. Horvitz, "Layered Representation for Learning and Inferring Office Activity from Multiple Sensory Channels," Fourth IEEE Conference on Multimodal Interfaces, pp. 3-8, 2002.Google ScholarDigital Library
- N. M. Oliver, B. Rosario, A. P. Pentland, "A Bayesian Computer Vision System for Modeling Human Interactions," IEEE Transactions of Pattern Analysis and Machine Intelligence, Vol. 22, pp. 831--843, 2000. Google ScholarDigital Library
- A. Pease and I. Niles, "Practical Semiotics: A Formal Theory". In Proceedings of the 2002 International Conference on Information and Knowledge Engineering, Las Vegas, Nevada, June 24--27, 2002.Google Scholar
- P. Perez, A. Blake, and M. Gangnet. "Jetstream: Probabilistic contour extraction with particles", Proc. of ICCV, pages 424-531, Vancouver, July 2001.Google ScholarCross Ref
- P. D. Sloane, C. M. Mitchell, K. Long and M. Lynn, "TESS 2+ Instrument B: Unit observation checklist - physical environment: A report on the psychometric properties of individual items, and initial recommendations on scaling", University of North Carolina 1995.Google Scholar
- J. F. Sowa, "Building, Sharing, and Merging Ontologies", web site: http://www.jfsowa.com/ontology/ontoshar.htm.Google Scholar
- D. Subrata, K. Shuster, and C. Wu, "Ontologies for Agent-Based Information Retrieval and Sequence Mining". In Proceedings of the Workshop on Ontologies in Agent Systems, International Joint Conference on Autonomous Agents and Multi-Agent Systems Bologna, Italy, July 15--19, 2002.Google Scholar
- Time Domain Corporation, 7057 Old Madison Pike, Huntsville, AL 35806. PulsON Technology: Time Modulated Ultra Wideband Overview, 2001.Google Scholar
- A. D. Wilson, A. F. Bobick, "Realtime Online Adaptive Gesture Recognition," International Workshop on Recognition, Analysis, and Tracking of Faces and Gestures in Real-Time Systems, pp. 111--117, 1999. Google ScholarDigital Library
- Y. Yacoob, M. J. Black, "Parameterized Modeling and Recognition of Activities," International Conference of Computer Vision, Vol. 73, pp. 232--247, 1998. Google ScholarDigital Library
- J. Yang, W. Lu, and A. Waibel, "Skin-color modeling and adaptation". In Proc. of ACCV, vol. II, pp. 687--694, 1998. Google ScholarDigital Library
- D. Zhang, S. Z. Li, D. Gatica-Perez, "Real-Time Face Detection Using Boosting Learning in Hierarchical Feature Spaces," 17th International Conference on Pattern Recognition 2004. Google ScholarDigital Library
Index Terms
- Towards automatic analysis of social interaction patterns in a nursing home environment from video
Recommendations
Detecting social interactions of the elderly in a nursing home environment
Social interaction plays an important role in our daily lives. It is one of the most important indicators of physical or mental changes in aging patients. In this article, we investigate the problem of detecting social interaction patterns of patients ...
Multimodal detection of human interaction events in a nursing home environment
ICMI '04: Proceedings of the 6th international conference on Multimodal interfacesIn this paper, we propose a multimodal system for detecting human activity and interaction patterns in a nursing home. Activities of groups of people are firstly treated as interaction patterns between any pair of partners and are then further broken ...
Designing an augmented print media system to promote social interaction in nursing homes: a preliminary study
ChineseCHI '18: Proceedings of the Sixth International Symposium of Chinese CHIPositive peer relationship in long term care facilities is an important component of residents' life quality. Hosting various activities in the public spaces of nursing homes has long been a typical and effective intervention to enhance residents' ...
Comments