Abstract
In this article, we consider how to automatically create pleasing photo collages created by placing a set of images on a limited canvas area. The task is formulated as an optimization problem. Differently from existing state-of-the-art approaches, we here exploit subjective experiments to model and learn pleasantness from user preferences. To this end, we design an experimental framework for the identification of the criteria that need to be taken into account to generate a pleasing photo collage. Five different thematic photo datasets are used to create collages using state-of-the-art criteria. A first subjective experiment where several subjects evaluated the collages, emphasizes that different criteria are involved in the subjective definition of pleasantness. We then identify new global and local criteria and design algorithms to quantify them. The relative importance of these criteria are automatically learned by exploiting the user preferences, and new collages are generated. To validate our framework, we performed several psycho-visual experiments involving different users. The results shows that the proposed framework allows to learn a novel computational model which effectively encodes an inter-user definition of pleasantness. The learned definition of pleasantness generalizes well to new photo datasets of different themes and sizes not used in the learning. Moreover, compared with two state-of-the-art approaches, the collages created using our framework are preferred by the majority of the users.
- Borji Ali, Cheng Ming-Ming, Jiang Huaizu, and Li Jia. 2014. Salient Object Detection: A Survey. (2014) arXiv:1411.5878 {cs.CV}.Google Scholar
- Sebastiano Battiato, Gianluigi Ciocca, Francesca Gasparini, Giovanni Puglisi, and Raimondo Schettini. 2008. Smart photo sticking. In Adaptive Multimedia Retrieval: Retrieval, User, and Semantics, Lecture Notes in Computer Science, vol. 4918. Springer, 211--223. Google ScholarDigital Library
- Simone Bianco and Raimondo Schettini. 2012. Sampling optimization for printer characterization by direct search. IEEE Trans. Image Process. 21, 12, 4868--4873. Google ScholarDigital Library
- Simone Bianco and Francesco Tisato. 2012. Sensor placement optimization in buildings. In Image Processing: Machine Vision Applications V, vol. 8300, SPIE, 830003.Google Scholar
- J. Calic, D. P. Gibson, and N. W. Campbell. 2007. Efficient layout of comic-like video summaries. IEEE Trans. Circ. Syst. Video Tech. 17, 7, 931--936. Google ScholarDigital Library
- Hui Chao, Daniel R. Tretter, Xuemei Zhang, and C. Brian Atkins. 2010. Blocked recursive image composition with exclusion zones. In Proceedings of the 10th ACM Symposium on Document Engineering (DocEng'10). ACM, 111--114. Google ScholarDigital Library
- Jun-Cheng Chen, Wei-Ta Chu, Jin-Hau Kuo, Chung-Yi Weng, and Ja-Ling Wu. 2006. Tiling slideshow. In Proceedings of the 14th Annual ACM International Conference on Multimedia (MULTIMEDIA'06). ACM, 25--34. Google ScholarDigital Library
- M. Cheng, N. J. Mitra, X. Huang, P. H. S. Torr, and S. Hu. 2015. Global contrast based salient region detection. IEEE Trans. Patt. Anal. Mach. Intell. 37, 3, 569--582.Google ScholarCross Ref
- Gianluigi Ciocca and Raimondo Schettini. 2010. Multiple image thumbnailing. In Digital Photography VI, vol. 7537, SPIE, 75370S.Google Scholar
- Nicholas Diakopoulos and Irfan Essa. 2005. Mediating photo collage authoring. In Proceedings of the 18th Annual ACM Symposium on User Interface Software and Technology (UIST'05). ACM, New York, 183--186. Google ScholarDigital Library
- K. Duncan and S. Sarkar. 2012. Saliency in images and video: A brief survey. Computer Vision, 6, 6, 514--523.Google ScholarCross Ref
- Russ C. Eberhart and James Kennedy. 1995. A new optimizer using particle swarm theory. In Proceedings of the 6th International Symposium on Micro Machine and Human Science. vol. 1, New York, 39--43.Google Scholar
- Hesam Ekhtiyar, Mahdi Sheida, and Mahmood Amintoosi. 2011. Picture collage with genetic algorithm and stereo vision. Int. J. Comput. Sci. Iss. 8, 4, 165--169.Google Scholar
- Jian Fan. 2012. Photo layout with a fast evaluation method and genetic algorithm. In Proceedings of the 2012 IEEE International Conference on Multimedia and Expo Workshops (ICMEW). IEEE, 308--313. Google ScholarDigital Library
- Andreas Girgensohn and Patrick Chiu. 2003. Stained glass photo collages. In Proceedings of the IEEE International Conference on Image Processing. vol. 2. 871--874.Google Scholar
- Stas Goferman, Ayellet Tal, and Lihi Zelnik-Manor. 2010. Puzzle-like Collage. Comput. Graph. Forum 29, 2, 459--468.Google ScholarCross Ref
- David E. Goldberg. 1989. Genetic Algorithms in Search, Optimization and Machine Learning (1st Ed.). Addison-Wesley Longman Publishing Co., Inc., Boston, MA. Google ScholarDigital Library
- Robert Hooke and To A. Jeeves. 1961. “Direct search” solution of numerical and statistical problems. J. ACM 8, 2, 212--229. Google ScholarDigital Library
- Hua Huang, Lei Zhang, and Hong-Chao Zhang. 2011. Arcimboldo-like collage using internet images. ACM Trans. Graph. 30, 6, 155:1--155:8. Google ScholarDigital Library
- Phillip Isola, Jianxiong Xiao, Antonio Torralba, and Aude Oliva. 2011. What makes an image memorable?. In Proceedings of the 2011 IEEE Conference on Computer Vision and Pattern Recognition (CVPR). IEEE, 145--152. Google ScholarDigital Library
- Maurice G. Kendall. 1938. A new measure of rank correlation. Biometrika 30, 1/2, 81--93.Google Scholar
- Aditya Khosla, Jianxiong Xiao, Antonio Torralba, and Aude Oliva. 2012. Memorability of image regions. In NIPS, Vol. 2. 4.Google Scholar
- Akisato Kimura, Ryo Yonetani, and Takatsugu Hirayama. 2013. Computational models of human visual attention and their implementations: A survey. IEICE Trans. Inf. Syst. 96, 3, 562--578.Google ScholarCross Ref
- Tamara G. Kolda, Robert Michael Lewis, and Virginia Torczon. 2003. Optimization by direct search: New perspectives on some classical and modern methods. SIAM Review 45, 3, 385--482.Google ScholarCross Ref
- Man Hee Lee, Nitin Singhal, Sungdae Cho, and In Kyu Park. 2010. Mobile photo collage. In Proceedings of the 2010 IEEE Conference on Computer Vision and Pattern Recognition Workshops (CVPRW). 24--30.Google ScholarCross Ref
- Tie Liu, Jingdong Wang, Jian Sun, Nanning Zheng, Xiaoou Tang, and Heung-Yeung Shum. 2009. Picture collage. IEEE Trans. Multimed. 11, 7, 1225--1239. Google ScholarDigital Library
- Sheng-Jie Luo, Chun-Yu Tsai, Wei-Chao Chen, and Bing-Yu Chen. 2013. Dynamic media assemblage. IEEE Trans. Circ. Syst. Video Tech. 23, 12, 2044--2053. Google ScholarDigital Library
- Yu-Fei Ma and Hong-Jiang Zhang. 2003. Contrast-based image attention analysis by using fuzzy growing. In Proceedings of the 11th ACM International Conference on Multimedia (MULTIMEDIA'03). ACM, 374--381. Google ScholarDigital Library
- Tao Mei, Bo Yang, Shi-Qiang Yang, and Xian-Sheng Hua. 2009. Video collage: Presenting a video sequence using a single image. Visual Computer 25, 1, 39--51. Google ScholarDigital Library
- A. Mittal, A. K. Moorthy, and A. C. Bovik. 2012. No-reference image quality assessment in the spatial domain. IEEE Trans. Image Process. 21, 12, 4695--4708. Google ScholarDigital Library
- John A. Nelder and Roger Mead. 1965. A simplex method for function minimization. Computer J. 7, 4, 308--313.Google ScholarCross Ref
- Li-Chen Ou and M. Ronnier Luo. 2006. A colour harmony model for two-colour combinations. Color Res. Appl. 31, 3, 191--204.Google ScholarCross Ref
- Carsten Rother, Lucas Bordeaux, Youssef Hamadi, and Andrew Blake. 2006. Autocollage. In ACM Trans. Graphics. vol. 25. ACM, New York, 847--852. Google ScholarDigital Library
- Carsten Rother, Sanjiv Kumar, Vladimir Kolmogorov, and Andrew Blake. 2005. Digital tapestry {automatic image synthesis}. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR'05). vol. 1. IEEE, 589--596. Google ScholarDigital Library
- Philipp Sandhaus, Mohammad Rabbath, and Susanne Boll. 2011. Employing aesthetic principles for automatic photo book layout. In Advances in Multimedia Modeling, Lecture Notes in Computer Science, vol. 6523, Springer, Berlin Heidelberg, 84--95. Google ScholarDigital Library
- Martin Solli and Reiner Lenz. 2009. Color harmony for image indexing. In Proceedings of the 2009 IEEE 12th International Conference on Computer Vision Workshops (ICCV Workshops), 1885--1892.Google ScholarCross Ref
- Jingdong Wang, Long Quan, Jian Sun, Xiaoou Tang, and Heung-Yeung Shum. 2006. Picture collage. In Proceedings of the 2006 IEEE Conference on Computer Vision and Pattern Recognition. vol. 1. IEEE, 347--354.Google Scholar
- Yichen Wei, Yasuyuki Matsushita, and Yingzhen Yang. 2009. Efficient optimization of photo collage. Tech. Rep. MSRTR-2009-59, Microsoft Research.Google Scholar
- Zhipeng Wu and K. Aizawa. 2013. PicWall: Photo collage on-the-fly. In Proceedings of the 2013 Asia -- Pacific Signal and Information Processing Association Annual Summit and Conference (APSIPA). IEEE, 1--10.Google Scholar
- Zhipeng Wu and Kiyoharu Aizawa. 2014. Building friend wall for local photo repository by using social attribute annotation. J. Multimedia 9, 1, 4--13.Google ScholarCross Ref
- Yingzhen Yang, Yichen Wei, Chunxiao Liu, Qunsheng Peng, and Yasuyuki Matsushita. 2009. An improved belief propagation method for dynamic collage. Visual Computer 25, 5--7, 431--439. Google ScholarDigital Library
- Zongqiao Yu, Lin Lu, Yanwen Guo, Rongfei Fan, Mingming Liu, and Wenping Wang. 2014. Content-aware photo collage using circle packing. IEEE Trans. Visual. Comput. Graph. 20, 2, 182--195. Google ScholarDigital Library
Index Terms
- User Preferences Modeling and Learning for Pleasing Photo Collage Generation
Recommendations
Mediating photo collage authoring
UIST '05: Proceedings of the 18th annual ACM symposium on User interface software and technologyThe medium of collage supports the visualization of meaningful event summaries using photographs. It can however be rather tedious to author a collage from a large collection of photographs. In this work we present an approach that supports efficient ...
Pseudo-3D photo collage
SIGGRAPH '02: ACM SIGGRAPH 2002 conference abstracts and applicationsPseudo-3D photo collage is a new technique for creating extensive pseudo-3D scenes on the Web. This technique enables users to create, publish, navigate and share pseudo-3D scenes by easy operations. Our basic idea comes from an artistic representation "...
Aesthetic Photo Collage With Deep Reinforcement Learning
Photo collage aims to automatically arrange multiple photos on a given canvas with high aesthetic quality. Existing methods are based mainly on handcrafted feature optimization, which cannot adequately capture high-level human aesthetic senses. Deep ...
Comments