Skip to main content
SpringerLink
Log in
Book cover

International Conference on Multimedia Modeling

MMM 2019: MultiMedia Modeling pp 361–373Cite as

Image Aesthetics Assessment Using Fully Convolutional Neural Networks

  • Conference paper
  • First Online:

Part of the book series: Lecture Notes in Computer Science ((LNISA,volume 11295))

Abstract

This paper presents a new method for assessing the aesthetic quality of images. Based on the findings of previous works on this topic, we propose a method that addresses the shortcomings of existing ones, by: (a) Making possible to feed higher-resolution images in the network, by introducing a fully convolutional neural network as the classifier. (b) Maintaining the original aspect ratio of images in the input of the network, to avoid distortions caused by re-scaling. And (c) combining local and global features from the image for making the assessment of its aesthetic quality. The proposed method is shown to achieve state of the art results on a standard large-scale benchmark dataset.

This is a preview of subscription content, log in via an institution.

Chapter
USD   29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD   39.99
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD   54.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Learn about institutional subscriptions

Notes

  1. 1.

    https://keras.io/.

  2. 2.

    Implementation of fully convolutional networks in Keras is available at https://github.com/bmezaris/fully_convolutional_networks.

References

  1. Bhattacharya, S., Sukthankar, R., Shah, M.: A framework for photo-quality assessment and enhancement based on visual aesthetics. In: Proceedings of 18th ACM International Conference on Multimedia (MM), pp. 271–280. ACM (2010)

    Google Scholar 

  2. Bianco, S., Celona, L., Napoletano, P., Schettini, R.: Predicting image aesthetics with deep learning. In: Blanc-Talon, J., Distante, C., Philips, W., Popescu, D., Scheunders, P. (eds.) ACIVS 2016. LNCS, vol. 10016, pp. 117–125. Springer, Cham (2016). https://doi.org/10.1007/978-3-319-48680-2_11

    Chapter  Google Scholar 

  3. Cui, C., Fang, H., Deng, X., Nie, X., Dai, H., Yin, Y.: Distribution-oriented aesthetics assessment for image search. In: Proceedings of 40th International SIGIR Conference on Research and Development in Information Retrieval, pp. 1013–1016. ACM (2017)

    Google Scholar 

  4. Deng, X., Cui, C., Fang, H., Nie, X., Yin, Y.: Personalized image aesthetics assessment. In: Proceedings of Conference on Information and Knowledge Management, pp. 2043–2046. ACM (2017)

    Google Scholar 

  5. Donahue, J., et al.: DeCAF: a deep convolutional activation feature for generic visual recognition. In: Proceedings of International Conference on Machine Learning (ICML), pp. 647–655 (2014)

    Google Scholar 

  6. Dong, Z., Shen, X., Li, H., Tian, X.: Photo quality assessment with DCNN that understands image well. In: He, X., Luo, S., Tao, D., Xu, C., Yang, J., Hasan, M.A. (eds.) MMM 2015. LNCS, vol. 8936, pp. 524–535. Springer, Cham (2015). https://doi.org/10.1007/978-3-319-14442-9_57

    Chapter  Google Scholar 

  7. Dong, Z., Tian, X.: Multi-level photo quality assessment with multi-view features. Neurocomputing 168, 308–319 (2015)

    Article  Google Scholar 

  8. Drozdzal, M., Vorontsov, E., Chartrand, G., Kadoury, S., Pal, C.: The importance of skip connections in biomedical image segmentation. In: Carneiro, G., et al. (eds.) LABELS/DLMIA -2016. LNCS, vol. 10008, pp. 179–187. Springer, Cham (2016). https://doi.org/10.1007/978-3-319-46976-8_19

    Chapter  Google Scholar 

  9. Guo, Y., Liu, M., Gu, T., Wang, W.: Improving photo composition elegantly: considering image similarity during composition optimization. In: Computer Graphics Forum, vol. 31, pp. 2193–2202. Wiley Online Library (2012)

    Google Scholar 

  10. Huang, G., Liu, Z., Van Der Maaten, L., Weinberger, K.Q.: Densely connected convolutional networks. In: Proceedings of Conference on Computer Vision and Pattern Recognition (CVPR), vol. 1, p. 3 (2017)

    Google Scholar 

  11. Jin, X., Chi, J., Peng, S., Tian, Y., Ye, C., Li, X.: Deep image aesthetics classification using inception modules and fine-tuning connected layer. In: Proceedings of IEEE 8th International Conference on Wireless Communications & Signal Processing (WCSP), pp. 1–6. IEEE (2016)

    Google Scholar 

  12. Kong, S., Shen, X., Lin, Z., Mech, R., Fowlkes, C.: Photo aesthetics ranking network with attributes and content adaptation. In: Leibe, B., Matas, J., Sebe, N., Welling, M. (eds.) ECCV 2016. LNCS, vol. 9905, pp. 662–679. Springer, Cham (2016). https://doi.org/10.1007/978-3-319-46448-0_40

    Chapter  Google Scholar 

  13. Krizhevsky, A., Ilya, S., Hinton, G.: ImageNet classification with deep convolutional neural networks. In: Proceedings of Conference on Advances in Neural Information Processing Systems (NIPS), pp. 1097–1105. Curran Associates, Inc. (2012)

    Google Scholar 

  14. Lemarchand, F.: From computational aesthetic prediction for images to films and online videos. Avant 8, 69–78 (2017)

    Article  Google Scholar 

  15. Li, Y., Zhang, T., Liu, Z., Hu, H.: A concatenating framework of shortcut convolutional neural networks. arXiv preprint arXiv:1710.00974 (2017)

  16. Liang, M., Hu, X., Zhang, B.: Convolutional neural networks with intra-layer recurrent connections for scene labeling. In: Proceedings of Conference on Advances in Neural Information Processing Systems (NIPS), Red Hook, NY Curran, pp. 937–945 (2015)

    Google Scholar 

  17. Lo, K.Y., Liu, K.H., Chen, C.S.: Assessment of photo aesthetics with efficiency. In: Proceedings of 21st International Conference on Pattern Recognition (ICPR), pp. 2186–2189. IEEE (2012)

    Google Scholar 

  18. Long, J., Shelhamer, E., Darrell, T.: Fully convolutional networks for semantic segmentation. In: Proceedings of Conference on Computer Vision and Pattern Recognition (CVPR), pp. 3431–3440. IEEE (2015)

    Google Scholar 

  19. Lu, X., Lin, Z., Jin, H., Yang, J., Wang, J.Z.: Rapid: rating pictorial aesthetics using deep learning. In: Proceedings of 22nd ACM Internatioanl Conference on Multimedia (MM), pp. 457–466. ACM (2014)

    Google Scholar 

  20. Lu, X., Lin, Z., Jin, H., Yang, J., Wang, J.Z.: Rating image aesthetics using deep learning. IEEE Trans. Multimedia 17(11), 2021–2034 (2015)

    Article  Google Scholar 

  21. Mai, L., Jin, H., Liu, F.: Composition-preserving deep photo aesthetics assessment. In: Proceedings of Conference on Computer Vision and Pattern Recognition (CVPR), pp. 497–506. IEEE (2016)

    Google Scholar 

  22. Marchesotti, L., Perronnin, F., Larlus, D., Csurka, G.: Assessing the aesthetic quality of photographs using generic image descriptors. In: Proceedings of IEEE International Conference on Computer Vision (ICCV), pp. 1784–1791. IEEE (2011)

    Google Scholar 

  23. Matan, O., Burges, C.J., LeCun, Y., Denker, J.S.: Multi-digit recognition using a space displacement neural network. In: Proceedings of Conference on Advances in Neural Information Processing Systems (NIPS), pp. 488–495 (1992)

    Google Scholar 

  24. Mavridaki, E., Mezaris, V.: A comprehensive aesthetic quality assessment method for natural images using basic rules of photography. In: Proceedings of IEEE International Conference on Image Processing (ICIP), pp. 887–891. IEEE (2015)

    Google Scholar 

  25. Murray, N., Marchesotti, L., Perronnin, F.: AVA: a large-scale database for aesthetic visual analysis. In: Proceedings of Conference on Computer Vision and Pattern Recognition (CVPR), pp. 2408–2415. IEEE (2012)

    Google Scholar 

  26. Nejdl, W., Niederee, C.: Photos to remember, photos to forget. IEEE Trans. MultiMedia (TMM) 22(1), 6–11 (2015)

    Article  Google Scholar 

  27. Obrador, P., Anguera, X., de Oliveira, R., Oliver, N.: The role of tags and image aesthetics in social image search. In: Proceedings of 1st SIGMM Workshop on Social Media, pp. 65–72. ACM (2009)

    Google Scholar 

  28. Pittaras, N., Markatopoulou, F., Mezaris, V., Patras, I.: Comparison of fine-tuning and extension strategies for deep convolutional neural networks. In: Amsaleg, L., Guðmundsson, G.Þ., Gurrin, C., Jónsson, B.Þ., Satoh, S. (eds.) MMM 2017. LNCS, vol. 10132, pp. 102–114. Springer, Cham (2017). https://doi.org/10.1007/978-3-319-51811-4_9

    Chapter  Google Scholar 

  29. Ren, J., Shen, X., Lin, Z.L., Mech, R., Foran, D.J.: Personalized image aesthetics. In: Proceedings of IEEE International Conference on Computer Vision (ICCV), pp. 638–647. IEEE (2017)

    Google Scholar 

  30. Simonyan, K., Zisserman, A.: Very deep convolutional networks for large-scale image recognition. In: Proceedings of International Conference on Learning Representations (ICLR) (2015)

    Google Scholar 

  31. Su, H.H., Chen, T.W., Kao, C.C., Hsu, W.H., Chien, S.Y.: Preference-aware view recommendation system for scenic photos based on bag-of-aesthetics-preserving features. IEEE Trans. Multimedia (TMM) 14(3), 833–843 (2012)

    Article  Google Scholar 

  32. Szegedy, C., et al.: Going deeper with convolutions. In: Proceedings of Conference on Computer Vision and Pattern Recognition (CVPR), pp. 1–9. IEEE (2015)

    Google Scholar 

  33. Wang, W., Zhao, M., Wang, L., Huang, J., Cai, C., Xu, X.: A multi-scene deep learning model for image aesthetic evaluation. Sig. Process. Image Commun. 47, 511–518 (2016)

    Article  Google Scholar 

  34. Wang, Z., Liu, D., Chang, S., Dolcos, F., Beck, D., Huang, T.: Image aesthetics assessment using deep Chatterjee’s machine. In: Proceedings of IEEE International Joint Conference on Neural Networks (IJCNN), pp. 941–948. IEEE (2017)

    Google Scholar 

  35. Yeh, C.H., Ho, Y.C., Barsky, B.A., Ouhyoung, M.: Personalized photograph ranking and selection system. In: Proceedings of 18th ACM International Conference on Multimedia (MM), pp. 211–220. ACM (2010)

    Google Scholar 

Download references

Acknowledgments

This work was supported by the EU’s Horizon 2020 research and innovation programme under contracts H2020-687786 InVID and H2020-732665 EMMA.

Author information

Authors and Affiliations

  1. Information Technologies Institute/CERTH, 6th km Charilaou - Thermi Road, Thermi, Thessaloniki, Greece

    Konstantinos Apostolidis & Vasileios Mezaris

Authors
  1. Konstantinos Apostolidis
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Vasileios Mezaris
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Vasileios Mezaris .

Editor information

Editors and Affiliations

  1. Information Technologies Institute, Centre for Research and Technology Hellas, Thessaloniki, Greece

    Ioannis Kompatsiaris

  2. EURECOM, Sophia Antipolis, France

    Benoit Huet

  3. Information Technologies Institute, Centre for Research and Technology Hellas, Thessaloniki, Greece

    Vasileios Mezaris

  4. Dublin City University, Dublin, Ireland

    Cathal Gurrin

  5. National Chiao Tung University, Hsinchu, Taiwan

    Wen-Huang Cheng

  6. Information Technologies Institute, Centre for Research and Technology Hellas, Thessaloniki, Greece

    Stefanos Vrochidis

Rights and permissions

Reprints and permissions

Copyright information

© 2019 Springer Nature Switzerland AG

About this paper

Check for updates. Verify currency and authenticity via CrossMark

Cite this paper

Apostolidis, K., Mezaris, V. (2019). Image Aesthetics Assessment Using Fully Convolutional Neural Networks. In: Kompatsiaris, I., Huet, B., Mezaris, V., Gurrin, C., Cheng, WH., Vrochidis, S. (eds) MultiMedia Modeling. MMM 2019. Lecture Notes in Computer Science(), vol 11295. Springer, Cham. https://doi.org/10.1007/978-3-030-05710-7_30

Download citation

  • DOI: https://doi.org/10.1007/978-3-030-05710-7_30

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-030-05709-1

  • Online ISBN: 978-3-030-05710-7

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics

Search

Navigation