Abstract
Aesthetic assessment is subjective, and the distribution of the aesthetic grades is over-concentrated in the middle levels. In order to realize the auto-assessment of photo aesthetics, we focus on using repetitive self-revised learning (RSRL) to retrain the convolutional neural network (CNN)-based aesthetics prediction network repetitively by the transfer learning, so as to improve the performance of imbalanced learning caused by the overconcentration distribution of aesthetic scores utilized as learning data. As RSRL, the network is trained repetitively by dropping out the low likelihood photo samples at the middle levels of aesthetics from the training data set based on the previously trained network. Further, the two retained networks are used in extracting aesthetic highlight elements of the photos to analyze the relation of the photo composition with the aesthetic assessment. The objective and subjective experimental results show that the CNN-based RSRL is effective for improving the performances of the imbalanced scores prediction network for the photos aesthetic auto-assessment.
This is a preview of subscription content, log in via an institution to check access.
Similar content being viewed by others
References
Colin G, McCormack JJ, Santos I, Romero J (2019) Understanding aesthetics and fitness measures in evolutionary art systems. Hidawi Complexity 3495962:1–14. https://doi.org/10.1155/2019/3495962
Dong Z, Tian X (November 2015) Multi-level photo quality assessment with multi-view features. Neurocomputing 168(30):308–319
He H, Garcia EA (2009) Learning from imbalanced data. TKDE 21(9):1263–1284
Huang C, et al. (2016) Learning deep representation for imbalanced classification”, 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), https://doi.org/10.1109/CVPR.2016.580, USA
Kao Y et al (September 2016) Hierarchical aesthetic quality assessment using deep convolutional neural networks. Signal Process Image Commun 47:500–510
Lemarchand F (September 2018) Fundamental visual features for aesthetic classification of photographs across datasets. Pattern Recognition Letters 112(1):9–17
Mavridaki E, Mezaris V (2015) A comprehensive aesthetic quality assessment method for natural images using basic rules of photography, 2015 IEEE international conference on image processing (ICIP) , Canada
Murray N, et al. (2012) AVA: a large-scale database for aesthetic visual analysis. Proceedings of the IEEE conference on computer vision and pattern recognition, Pages 2408–2415
Tan Y, Tang P, Zhou Y, Luo W, Kang Y, Li G (March 2017) Photograph aesthetical evaluation and classification with deep convolutional neural networks. Neurocomputing 228(8):165–175
Wang W et al (September 2016) A multi-scene deep learning model for image aesthetic evaluation. Signal Process Image Commun 47:511–518
Yubin, Chen Change Loy, and Xiaoou Tang, “Image aesthetic assessment: an experimental survey”, arXiv: 1610.00838v2
Zhang C et al (September 2018) Visual aesthetic understanding: sample-specific aesthetic classification and deep activation map visualization. Signal Process Image Commun 67:12–21
Author information
Authors and Affiliations
Corresponding author
Additional information
Publisher’s note
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Rights and permissions
About this article
Cite this article
Dai, Y. Sample-specific repetitive learning for photo aesthetic auto-assessment and highlight elements analysis. Multimed Tools Appl 80, 1387–1402 (2021). https://doi.org/10.1007/s11042-020-09426-z
Received:
Revised:
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s11042-020-09426-z