Skip to main content
SpringerLink
Log in
Book cover

International Conference on Intelligent Data Engineering and Automated Learning

IDEAL 2017: Intelligent Data Engineering and Automated Learning – IDEAL 2017 pp 387–396Cite as

Markov Random Field Based Convolutional Neural Networks for Image Classification

  • Conference paper
  • First Online:

Part of the book series: Lecture Notes in Computer Science ((LNISA,volume 10585))

Abstract

In image classification, deriving efficient image representations from raw data is a key focus as it can largely determine the performance of a vision system. Conventional methods extract low-level features based on experiments or certain theories, whilst deep learning approaches learn image representations hierarchically with multiple layers of abstraction from vast number of sample images. Markov random fields are generative, flexible and stochastic image texture models, in which global image representations can be obtained by means of local conditional probabilities. Texture has been strongly linked to human visual perception. The ability of deriving global description from local structure shares compatibility with convolutional neural networks. Inspired by this property, we investigate the combination of Markov random field models with deep convolutional neural networks for image classification. Various filters from Markov random field models are first derived to form the features maps. Then convolutional neural networks are trained with prefixed filter banks. Comprehensive experiments conducted on the MNIST dataset, EMNIST database and CIFAR-10 object database are reported.

This is a preview of subscription content, log in via an institution.

Chapter
USD   29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD   39.99
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD   54.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Learn about institutional subscriptions

References

  1. Bengio, Y., Courville, A., Vincent, P.: Representation learning: a review and new perspectives. IEEE Trans. Pattern Anal. Mach. Intell. 35(8), 1798–1828 (2013)

    Article  Google Scholar 

  2. Hinton, G.E., Osindero, S., Teh, Y.W.: A fast learning algorithm for deep belief nets. Neural Comput. 18(7), 1527–1554 (2006)

    Article  MathSciNet  MATH  Google Scholar 

  3. LeCun, Y., Bengio, Y., Hinton, G.E.: Deep learning. Nature 521(7553), 436–444 (2015)

    Article  Google Scholar 

  4. LeCun, Y., Kavukcuoglu, K., Farabet, C.: Convolutional networks and applications in vision. In: Proceedings of IEEE International Symposium on Circuits and Systems, pp. 253–256 (2010)

    Google Scholar 

  5. Julesz, B.: Visual pattern discrimination. IRE Trans. Inf. Theor. 8(2), 84–92 (1962)

    Article  Google Scholar 

  6. Julesz, B.: Textons, the elements of texture perception, and their interactions. Nature 290(5802), 91–97 (1981)

    Article  Google Scholar 

  7. Li, S.Z.: A Markov random field model for object matching under contextual constraints. In: Proceedings of IEEE Computer Society Conference on Computer Vision and Pattern Recognition, p. 866 (1994)

    Google Scholar 

  8. Li, S.Z.: Modeling image analysis problems using Markov random fields. Stochast. Processes Model. Simul. 20(5), 1–43 (2000)

    Google Scholar 

  9. Geman, S., Geman, D.: Stochastic relaxation, gibbs distributions, and the Bayesian restoration of images. IEEE Trans. Pattern Anal. Mach. Intell. 6(6), 721–741 (1984)

    Article  MATH  Google Scholar 

  10. LeCun, Y., Bottou, L., Bengio, Y., Haffner, P.: Gradient-based learning applied to document recognition. Proc. IEEE 86(11), 2278–2324 (1998)

    Article  Google Scholar 

  11. Cohen, G., Afshar, S., Tapson, J., van Schaik, A.: Emnist: an extension of mnist to handwritten letters (2017). arXiv preprint: arXiv:1702.05373

  12. Krizhevsky, A., Hinton, G.E.: Learning multiple layers of features from tiny images. Master Thesis, the University of Toronto (2009)

    Google Scholar 

  13. Cross, G.R., Jain, A.K.: Markov random field texture models. IEEE Trans. Pattern Anal. Mach. Intell. 5(1), 25–39 (1983)

    Article  Google Scholar 

  14. Ising, E.: Beitrag zur theorie des ferromagnetismus. Zeitschrift für Physik 31(1), 253–258 (1925)

    Article  Google Scholar 

  15. Yin, H., Allinson, N.M.: Unsupervised segmentation of textured images using a hierarchical neural structure. Electron. Lett. 30(22), 1842–1843 (1994)

    Article  Google Scholar 

  16. Nishii, R., Eguchi, S.: Image classification based on Markov random field models with jeffreys divergence. J. Multivar. Anal. 97(9), 1997–2008 (2006)

    Article  MathSciNet  MATH  Google Scholar 

  17. Krizhevsky, A., Sutskever, I., Hinton, G.E.: Imagenet classification with deep convolutional neural networks. In: Advances in Neural Information Processing Systems, pp. 1097–1105 (2012)

    Google Scholar 

  18. Zeiler, M.D., Fergus, R.: Visualizing and understanding convolutional networks. In: Fleet, D., Pajdla, T., Schiele, B., Tuytelaars, T. (eds.) ECCV 2014. LNCS, vol. 8689, pp. 818–833. Springer, Cham (2014). doi:10.1007/978-3-319-10590-1_53

    Google Scholar 

  19. Simonyan, K., Zisserman, A.: Very deep convolutional networks for large-scale image recognition (2014). arXiv preprint: arXiv:1409.1556

  20. Szegedy, C., Liu, W., Jia, Y., Sermanet, P., Reed, S., Anguelov, D., Erhan, D., Vanhoucke, V., Rabinovich, A.: Going deeper with convolutions. In: Proceedings of the IEEE International Conference on Computer Vision and Pattern Recognition, pp. 1–9 (2015)

    Google Scholar 

  21. Cireşan, D., Meier, U., Schmidhuber, J.: Multi-column deep neural networks for image classification. In: Proceedings of the IEEE International Conference on Computer Vision and Pattern Recognition, pp. 3642–3649. IEEE (2012)

    Google Scholar 

  22. Jarrett, K., Kavukcuoglu, K., LeCun, Y., et al.: What is the best multi-stage architecture for object recognition? In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2146–2153 (2009)

    Google Scholar 

  23. Lin, M., Chen, Q., Yan, S.: Network in network (2013) arXiv preprint: arXiv:1312.4400

  24. Zeiler, M.D., Fergus, R.: Stochastic pooling for regularization of deep convolutional neural networks (2013). arXiv preprint: arXiv:1301.3557

  25. Springenberg, J.T., Dosovitskiy, A., Brox, T., Riedmiller, M.: Striving for simplicity: the all convolutional net (2014). arXiv preprint: arXiv:1412.6806

  26. Bruna, J., Mallat, S.: Invariant scattering convolution networks. IEEE Trans. Pattern Anal. Mach. Intell. 35(8), 1872–1886 (2013)

    Article  Google Scholar 

  27. Chan, T.H., Jia, K., Gao, S., Lu, J., Zeng, Z., Ma, Y.: PCANet: a simple deep learning baseline for image classification? IEEE Trans. Image Process. 24(12), 5017–5032 (2015)

    Article  MathSciNet  Google Scholar 

  28. Ng, C.J., Teoh, A.B.J.: Dctnet: a simple learning-free approach for face recognition. In: Proceedings of Asia-Pacific Conference on Signal and Information Processing Association Annual Summit, pp. 761–768 (2015)

    Google Scholar 

  29. Besag, J.: Spatial interaction and the statistical analysis of lattice systems. J. R. Stat. Soc. Ser. B 36, 192–236 (1974)

    MathSciNet  MATH  Google Scholar 

  30. Hammersley, J.M., Clifford, P.E.: Markov random fields on finite graphs and lattices. Unpublished manuscript (1971)

    Google Scholar 

  31. Elliott, H., Derin, H., Cristi, R., Geman, D.: Application of the gibbs distribution to image segmentation. In: Proceedings of IEEE International Conference on Acoustics, Speech, and Signal Processing, vol. 9, pp. 678–681 (1984)

    Google Scholar 

  32. Derin, H., Elliott, H.: Modeling and segmentation of noisy and textured images using gibbs random fields. IEEE Trans. Pattern Anal. Mach. Intell. 9(1), 39–55 (1987)

    Article  Google Scholar 

  33. Kashyap, R., Chellappa, R.: Estimation and choice of neighbors in spatial-interaction models of images. IEEE Trans. Inf. Theor. 29(1), 60–72 (1983)

    Article  MATH  Google Scholar 

  34. Dass, S.C.: Markov random field models for directional field and singularity extraction in fingerprint images. IEEE Trans. Image Process. 13(10), 1358–1367 (2004)

    Article  Google Scholar 

  35. Goodfellow, I.J., Warde-Farley, D., Mirza, M., Courville, A., Bengio, Y.: Maxout networks (2013). arXiv preprint: arXiv:1302.4389

  36. Wan, L., Zeiler, M., Zhang, S., Cun, Y.L., Fergus, R.: Regularization of neural networks using dropconnect. In: Proceedings of the International Conference on Machine Learning, pp. 1058–1066 (2013)

    Google Scholar 

Download references

Author information

Authors and Affiliations

  1. School of Electrical and Electronic Engineering, The University of Manchester, Manchester, M13 9PL, UK

    Yao Peng & Hujun Yin

Authors
  1. Yao Peng
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Hujun Yin
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Yao Peng .

Editor information

Editors and Affiliations

  1. University of Manchester, Manchester, United Kingdom

    Hujun Yin

  2. School of Electronic and Electrical Engineering, Nanjing University, Nanjiing, China

    Yang Gao

  3. Nanjing University of Aeronautics and Astronautics, Nanjing, China

    Songcan Chen

  4. Guilin University of Electronic Technology, Guilin, China

    Yimin Wen

  5. Guilin University of Electronic Technology, Guilin, China

    Guoyong Cai

  6. Guilin University of Electronic Technology, Guilin, China

    Tianlong Gu

  7. Beijing University of Posts and Telecommunications, Beijing, China

    Junping Du

  8. University of Seville, Seville, Spain

    Antonio J. Tallón-Ballesteros

  9. Southeast University, Nanjing, China

    Minling Zhang

Rights and permissions

Reprints and permissions

Copyright information

© 2017 Springer International Publishing AG

About this paper

Cite this paper

Peng, Y., Yin, H. (2017). Markov Random Field Based Convolutional Neural Networks for Image Classification. In: Yin, H., et al. Intelligent Data Engineering and Automated Learning – IDEAL 2017. IDEAL 2017. Lecture Notes in Computer Science(), vol 10585. Springer, Cham. https://doi.org/10.1007/978-3-319-68935-7_42

Download citation

  • DOI: https://doi.org/10.1007/978-3-319-68935-7_42

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-319-68934-0

  • Online ISBN: 978-3-319-68935-7

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics

Search

Navigation