Skip to main content
SpringerLink
Log in

Multi-Instance Dynamic Ordinal Random Fields for Weakly-Supervised Pain Intensity Estimation

  • Conference paper
  • First Online:
Computer Vision – ACCV 2016 (ACCV 2016)

Part of the book series: Lecture Notes in Computer Science ((LNIP,volume 10112))

Included in the following conference series:

Abstract

In this paper, we address the Multi-Instance-Learning (MIL) problem when bag labels are naturally represented as ordinal variables (Multi-Instance-Ordinal Regression). Moreover, we consider the case where bags are temporal sequences of ordinal instances. To model this, we propose the novel Multi-Instance Dynamic Ordinal Random Fields (MI-DORF). In this model, we treat instance-labels inside the bag as latent ordinal states. The MIL assumption is modelled by incorporating a high-order cardinality potential relating bag and instance-labels, into the energy function. We show the benefits of the proposed approach on the task of weakly-supervised pain intensity estimation from the UNBC Shoulder-Pain Database. In our experiments, the proposed approach significantly outperforms alternative non-ordinal methods that either ignore the MIL assumption, or do not model dynamic information in target data.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 39.99
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 54.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

Notes

  1. 1.

    Total number of observations T can vary across different sequences.

  2. 2.

    The potential with the Multinomial Logistic Regession model is defined as \(\log ( \frac{ \exp (\beta ^T_l x)}{ \sum _{ l^\prime \in L} \exp (\beta ^T_{l^\prime } x) } )\). Where all \(\mathbf {\beta _l}\) defines a linear projection for each possible ordinal value l [32].

References

  1. Babenko, B., Yang, M.H., Belongie, S.: Robust object tracking with online multiple instance learning. IEEE Trans. Pattern Anal. Mach. Intell. 33, 1619–1632 (2011)

    Article  Google Scholar 

  2. Wu, J., Zhao, Y., Zhu, J.Y., Luo, S., Tu, Z.: Milcut: a sweeping line multiple instance learning paradigm for interactive image segmentation. In: Proceedings of the Computer Vision and Pattern Recognition (2014)

    Google Scholar 

  3. Ruiz, A., Van de Weijer, J., Binefa, X.: Regularized multi-concept MIL for weakly-supervised facial behavior categorization. In: Proceedings of the British Machine Vision Conference (2014)

    Google Scholar 

  4. Maron, O., Lozano-Pérez, T.: A framework for multiple-instance learning. In: Advances in Neural Information Processing Systems (1998)

    Google Scholar 

  5. Ray, S., Page, D.: Multiple instance regression. In: Proceedings of the International Conference on Machine Learning (2001)

    Google Scholar 

  6. Lucey, P., Cohn, J.F., Prkachin, K.M., Solomon, P.E., Matthews, I.: Painful data: the UNBC-McMaster shoulder pain expression archive database. In: International Conference on Automatic Face and Gesture Recognition (2011)

    Google Scholar 

  7. Aung, M.S., Kaltwang, S., Romera-Paredes, B., Martinez, B., Singh, A., Cella, M., Valstar, M.F., Meng, H., Kemp, A., Shafizadeh, M., Elkins, A.C., Kanakam, N., rothschild, A.D., Tyler, N., Watson, P.J., Williams, A.C., Pantic, M., Bianchi-berthouze, N.: The automatic detection of chronic pain-related expression: requirements, challenges and a multimodal dataset. IEEE Trans. Affect. Comput. (2015, to appear)

    Google Scholar 

  8. Hjermstad, M.J., Fayers, P.M., Haugen, D.F., Caraceni, A., Hanks, G.W., Loge, J.H., Fainsinger, R., Aass, N., Kaasa, S., EPCRC, E.P.C.R.C., et al.: Studies comparing numerical rating scales, verbal rating scales, and visual analogue scales for assessment of pain intensity in adults: a systematic literature review. J. Pain Symptom Manag. 41, 1073–1093 (2011)

    Google Scholar 

  9. Rudovic, O., Pavlovic, V., Pantic, M.: Automatic pain intensity estimation with heteroscedastic conditional ordinal random fields. In: Bebis, G., et al. (eds.) ISVC 2013. LNCS, vol. 8034, pp. 234–243. Springer, Heidelberg (2013). doi:10.1007/978-3-642-41939-3_23

    Chapter  Google Scholar 

  10. Kaltwang, S., Rudovic, O., Pantic, M.: Continuous pain intensity estimation from facial expressions. In: Bebis, G., et al. (eds.) ISVC 2012. LNCS, vol. 7432, pp. 368–377. Springer, Heidelberg (2012). doi:10.1007/978-3-642-33191-6_36

    Chapter  Google Scholar 

  11. Wu, C., Wang, S., Ji, Q.: Multi-instance hidden Markov model for facial expression recognition. In: International Conference on Automatic Face and Gesture Recognition (2015)

    Google Scholar 

  12. Sikka, K., Dhall, A., Bartlett, M.: Weakly supervised pain localization using multiple instance learning. In: International Conference on Automatic Face and Gesture Recognition (2013)

    Google Scholar 

  13. Kim, M., Pavlovic, V.: Hidden conditional ordinal random fields for sequence classification. In: Balcázar, J.L., Bonchi, F., Gionis, A., Sebag, M. (eds.) ECML PKDD 2010. LNCS (LNAI), vol. 6322, pp. 51–65. Springer, Heidelberg (2010). doi:10.1007/978-3-642-15883-4_4

    Chapter  Google Scholar 

  14. Liu, J., Chen, C., Zhu, Y., Liu, W., Metaxas, D.N.: Video classification via weakly supervised sequence modeling. Comput. Vis. Image Underst. 152, 79–87 (2015)

    Article  Google Scholar 

  15. Barber, D.: Bayesian Reasoning and Machine Learning. Cambridge University Press, Cambridge (2012)

    MATH  Google Scholar 

  16. Amores, J.: Multiple instance classification: review, taxonomy and comparative study. Artif. Intell. 201, 81–105 (2013)

    Article  MathSciNet  MATH  Google Scholar 

  17. Gärtner, T., Flach, P.A., Kowalczyk, A., Smola, A.J.: Multi-instance kernels. In: Proceedings of the International Conference on Machine Learning (2002)

    Google Scholar 

  18. Chen, Y., Bi, J., Wang, J.Z.: Miles: Multiple-instance learning via embedded instance selection. IEEE Trans. Pattern Anal. Mach. Intell. 28, 1931–1947 (2006)

    Article  Google Scholar 

  19. Zhou, Z.H., Sun, Y.Y., Li, Y.F.: Multi-instance learning by treating instances as non-I.I.D. samples. In: Proceedings of the International Conference on Machine Learning (2009)

    Google Scholar 

  20. Wagstaff, K.L., Lane, T., Roper, A.: Multiple-instance regression with structured data. In: International Conference on Data Mining (2008)

    Google Scholar 

  21. Andrews, S., Tsochantaridis, I., Hofmann, T.: Support vector machines for multiple-instance learning. In: Advances in Neural Information Processing Systems (2002)

    Google Scholar 

  22. Zhang, C., Platt, J.C., Viola, P.A.: Multiple instance boosting for object detection. In: Advances in Neural Information Processing Systems (2005)

    Google Scholar 

  23. Hsu, K.J., Lin, Y.Y., Chuang, Y.Y.: Augmented multiple instance regression for inferring object contours in bounding boxes. IEEE Trans. Image Process. 23, 1722–1736 (2014)

    Article  MathSciNet  Google Scholar 

  24. Hajimirsadeghi, H., Li, J., Mori, G., Zaki, M., Sayed, T.: Multiple instance learning by discriminative training of Markov networks. In: Uncertainty in Artificial Intelligence (2013)

    Google Scholar 

  25. Quattoni, A., Wang, S., Morency, L.P., Collins, M., Darrell, T.: Hidden conditional random fields. IEEE Trans. Pattern Anal. Mach. Intell. 29, 1848–1852 (2007)

    Article  Google Scholar 

  26. Rabiner, L.R., Juang, B.H.: An introduction to hidden Markov models. ASSP Mag. 3, 4–16 (1986)

    Article  Google Scholar 

  27. Winkelmann, R., Boes, S.: Analysis of Microdata. Springer Science Business Media, Berlin (2006)

    MATH  Google Scholar 

  28. Gupta, R., Diwan, A.A., Sarawagi, S.: Efficient inference with cardinality-based clique potentials. In: Proceedings of the International Conference on Machine Learning (2007)

    Google Scholar 

  29. Byrd, R.H., Nocedal, J., Schnabel, R.B.: Representations of quasi-newton matrices and their use in limited memory methods. Math. Program. 63, 129–156 (1994)

    Article  MathSciNet  MATH  Google Scholar 

  30. Tarlow, D., Swersky, K., Zemel, R.S., Adams, R.P.: Fast exact inference for recursive cardinality models. In: Conference on Uncertainty in Artificial Intelligence (2012)

    Google Scholar 

  31. Ray, S., Craven, M.: Supervised versus multiple instance learning: an empirical comparison. In: Proceedings of the International Conference on Machine Learning (2005)

    Google Scholar 

  32. Walecki, R., Rudovic, O., Pavlovic, V., Pantic, M.: Variable-state latent conditional random fields for facial expression recognition and action unit detection. In: International Conference on Automatic Face and Gesture Recognition (2015)

    Google Scholar 

  33. Prkachin, K.M.: The consistency of facial expressions of pain: a comparison across modalities. Pain 51, 297–306 (1992)

    Article  Google Scholar 

  34. Xuehan-Xiong, D., la Torre, F.: Supervised descent method and its application to face alignment. In: Proceedings of the Computer Vision and Pattern Recognition (2013)

    Google Scholar 

  35. Rudovic, O., Pavlovic, V., Pantic, M.: Context-sensitive dynamic ordinal regression for intensity estimation of facial action units. IEEE Trans. Pattern Anal. Mach. Intell. 37, 944–958 (2015)

    Article  Google Scholar 

Download references

Acknowledgement

This paper is part of a project that has received funding from the European Union’s Horizon 2020 research and innovation programme under grants agreement no. 645012 (KRISTINA), no. 645094 (SEWA) and no. 688835 (DE-ENIGMA). Adria Ruiz would also like to acknowledge Spanish Government to provide support under grant FPU13/01740.

Author information

Authors and Affiliations

  1. DTIC, Universitat Pompeu Fabra, Barcelona, Spain

    Adria Ruiz & Xavier Binefa

  2. Department of Computing, Imperial College London, London, UK

    Ognjen Rudovic & Maja Pantic

  3. EEMCS, University of Twente, Enschede, The Netherlands

    Maja Pantic

Authors
  1. Adria Ruiz
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Ognjen Rudovic
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Xavier Binefa
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Maja Pantic
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Adria Ruiz .

Editor information

Editors and Affiliations

  1. National Tsing Hua University, Hsinchu, Taiwan

    Shang-Hong Lai

  2. Graz University of Technology, Graz, Austria

    Vincent Lepetit

  3. Drexel University, Philadelphia, Pennsylvania, USA

    Ko Nishino

  4. The University of Tokyo, Tokyo, Japan

    Yoichi Sato

Rights and permissions

Reprints and permissions

Copyright information

© 2017 Springer International Publishing AG

About this paper

Cite this paper

Ruiz, A., Rudovic, O., Binefa, X., Pantic, M. (2017). Multi-Instance Dynamic Ordinal Random Fields for Weakly-Supervised Pain Intensity Estimation. In: Lai, SH., Lepetit, V., Nishino, K., Sato, Y. (eds) Computer Vision – ACCV 2016. ACCV 2016. Lecture Notes in Computer Science(), vol 10112. Springer, Cham. https://doi.org/10.1007/978-3-319-54184-6_11

Download citation

  • DOI: https://doi.org/10.1007/978-3-319-54184-6_11

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-319-54183-9

  • Online ISBN: 978-3-319-54184-6

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics

Search

Navigation