Abstract
On March 11, 1944, the famous Eremitani Church in Padua (Italy) was destroyed in an Allied bombing along with the inestimable frescoes by Andrea Mantegna et al. contained in the Ovetari Chapel. In the last 60 years, several attempts have been made to restore the fresco fragments by traditional methods, but without much success. We have developed an efficient pattern recognition algorithm to map the original position and orientation of the fragments, based on comparisons with an old gray level image of the fresco prior to the damage. This innovative technique allowed for the partial reconstruction of the frescoes. Unfortunately, the surface covered by the fragments is only 77 m 2, while the original area was of several hundreds. This means that we can reconstruct only a fraction (less than 8%) of this inestimable artwork. In particular the original color of the blanks is not known. This begs the question of whether it is possible to estimate mathematically the original colors of the frescoes by making use of the potential information given by the available fragments and the gray level of the pictures taken before the damage. Can one estimate how faithful such restoration is? In this paper we retrace the development of the recovery of the frescoes as an inspiring and challenging real-life problem for the development of new mathematical methods. We introduce two models for the recovery of vector valued functions from incomplete data, with applications to the fresco recolorization problem. The models are based on the minimization of a functional which is formed by the discrepancy with respect to the data and additional regularization constraints. The latter refer to joint sparsity measures with respect to frame expansions for the first functional and functional total variation for the second. We establish the relations between these two models. As a byproduct we develop the basis of a theory of fidelity in color recovery, which is a crucial issue in art restoration and compression.
This is a preview of subscription content, log in via an institution.
Buying options
Tax calculation will be finalised at checkout
Purchases are for personal use only
Learn about institutional subscriptionsPreview
Unable to display preview. Download preview PDF.
References
Ballester, C., et al.: Filling-in by joint interpolation of vector fields and gray levels. IEEE Trans. Image Process. 10(8), 1200–1211 (2001)
Baron, D., et al.: Distributed compressed sensing. Preprint (2005)
Beltramio, M., et al.: Image inpainting. In: SIGGRAPH 2000 (2001)
Candès, E.J., Donoho, D.L.: New tight frames of curvelets and optimal representations of objects with piecewise C 2 singularities. Commun. Pure Appl. Math. 57(2), 219–266 (2004)
Candès, E.J., Romberg, E.J., Tao, T.: Exact signal reconstruction from highly incomplete frequency information. IEEE Trans. Inf. Theory 52(2), 489–509 (2006)
Candès, E.J., Romberg, E.J., Tao, T.: Stable signal recovery from incomplete and inaccurate measurements. Preprint (2005)
Caselles, V., Coll, V., Morel, J.-M.: Geometry and color in natural images. Journal of Mathematical Imaging and Vision 16(2), 89–105 (2002)
Cazzato, R., et al.: Il Progetto Mantegna: storia e risultati. In: Spiazzi, A.M., De Nicolò Salmazo, A., Toniolo, D. (eds.) Andrea Mantegna e i Maestri della Cappella Ovetari: La Ricomposizione Virtuale e il Restauro, Skira, Milano (2006)
Chan, T.F., Kang, S.H.: Error analysis for image inpainting. UCLA CAM 04-72 (2004)
Chan, T.F., Kang, S.H., Shen, J.: Euler’s elastica and curvaure-based inpainting. SIAM J. Appl. Math. 63(2), 564–592 (2002)
Chan, T.F., Shen, J.: Inpainting based on nonlinear transport and diffusion. Contemp. Math. 313, 53–65 (2002)
Chan, T.F., Shen, J.: Mathematical models for local nontexture inpaintings. SIAM J. Appl. Math. 62(3), 1019–1043 (2002)
Chan, T.F., Shen, J.: Variational image inpainting. Commun. Pure Appl. Math. 58(5), 579–619 (2005)
Daubechies, I., Defrise, M., DeMol, C.: An iterative thresholding algorithm for linear inverse problems. Comm. Pure Appl. Math. 57(11), 1413–1457 (2004)
Donoho, D.L.: Compressed Sensing. IEEE Trans. Inf. Theory 52(4), 1289–1306 (2006)
Elad, M., et al.: Simultaneous cartoon and texture image inpainting using morphological component analysis (MCA). Appl. Comput. Harmon. Anal. 19, 340–358 (2005)
Fornasier, M.: Nonlinear projection recovery in digital inpainting for color image restoration. J. Math. Imaging Vis. 24(3), 359–373 (2006)
Fornasier, M., March, R.: Restoration of color images by vector valued BV functions and variational calculus. Johann Radon Institute for Computational and Applied Mathematics (RICAM), preprint 2006-30 (2006), http://www.ricam.oeaw.ac.at/publications/reports/06/rep06-30.pdf
Fornasier, M., Rauhut, H.: Recovery algorithms for vector valued data with joint sparsity constraints. Johann Radon Institute for Computational and Applied Mathematics (RICAM) preprint 2006-27 (2006), http://www.ricam.oeaw.ac.at/publications/reports/06/rep06-27.pdf
Fornasier, M., Toniolo, D.: Fast, robust, and efficient 2D pattern recognition for re-assembling fragmented digital images. Pattern Recognition 38, 2074–2087 (2005)
Gao, H.-Y., Bruce, A.G.: WaveShrink with firm shrinkage. Statist. Sinica 7(4), 855–874 (1997)
Kang, S.H., March, R.: Variational models for image colorization via Chromaticity and Brightness decomposition. Preprint (2006)
Levin, A., Lischinski, A., Weiss, Y.: Colorization using optimization. ACM Transactions on Graphics (Proceedings of the 2004 SIGGRAPH Conference) 23(3), 689–694 (2004)
Mallat, S.: A Wavelet Tour of Signal Processing, 2nd edn. Academic Press, San Diego (1999)
Masnou, S.: Filtrage et Désocclusion d’Images par Méthodes d’Ensembles de Niveau. PhD Thesis, Université Paris-Dauphine (1998)
Masnou, S.: Disocclusion: a variational approach using level lines. IEEE Trans. on Image Processing 11(2), 68–76 (2002)
Masnou, S., Morel, J.-M.: Level lines based disocclusion. In: Proceedings of 5th IEEE Int’l Conf. on Image Process, Chicago, vol. 3, pp. 259–263 (1998)
Mumford, D.: Elastica and computer vision. In: Bajaj, C. (ed.) Algebraic geometry and applications, pp. 491–506. Springer, Heidelberg (1994)
Sapiro, G.: Inpainting the colors. In: IEEE International Conference on Image Processing (ICIP 2005), vol. 2, pp. 698–701 (2005)
Tropp, J.: Algorithms for simultaneous sparse approximation. Part II: Convex relaxation. Signal Processing 86, 589–602 (2006)
Yatziv, L., Sapiro, G.: Fast image and video colorization using chrominance blending. IEEE Transactions on Image Processing 15(5), 1120–1129 (2006)
Author information
Authors and Affiliations
Editor information
Rights and permissions
Copyright information
© 2007 Springer Berlin Heidelberg
About this paper
Cite this paper
Fornasier, M. (2007). Faithful Recovery of Vector Valued Functions from Incomplete Data. In: Sgallari, F., Murli, A., Paragios, N. (eds) Scale Space and Variational Methods in Computer Vision. SSVM 2007. Lecture Notes in Computer Science, vol 4485. Springer, Berlin, Heidelberg. https://doi.org/10.1007/978-3-540-72823-8_11
Download citation
DOI: https://doi.org/10.1007/978-3-540-72823-8_11
Publisher Name: Springer, Berlin, Heidelberg
Print ISBN: 978-3-540-72822-1
Online ISBN: 978-3-540-72823-8
eBook Packages: Computer ScienceComputer Science (R0)