Abstract.
Recently, Field, Lewicki, Olshausen, and Sejnowski have reported efforts to identify the ``Sparse Components'' of image data. Their empirical findings indicate that such components have elongated shapes and assume a wide range of positions, orientations, and scales.
To date, sparse components analysis (SCA) has only been conducted on databases of small (e.g., 16 by 16) image patches and there seems limited prospect of dramatically increased resolving power. In this paper, we apply mathematical analysis to a specific formalization of SCA using synthetic image models, hoping to gain insight into what might emerge from a higher-resolution SCA based on n by n image patches for large n but a constant field of view.
In our formalization, we study a class of objects \cal F in a functional space; they are to be represented by linear combinations of atoms from an overcomplete dictionary, and sparsity is measured by the ℓ p -norm of the coefficients in the linear combination. We focus on the class \cal F = \sc Starα of black and white images with the black region consisting of a star-shaped set with an α -smooth boundary. We aim to find an optimal dictionary, one achieving the optimal sparsity in an atomic decomposition uniformly over members of the class \sc Starα .
We show that there is a well-defined optimal sparsity of representation of members of \sc Starα; there are decompositions with finite ℓ p -norm for p > 2/(α+1) but not for p < 2/(α+1) . We show that the optimal degree of sparsity is nearly attained using atomic decompositions based on the wedgelet dictionary.
Wedgelets provide a system of representation by elements in a dyadically organized collection, at all scales, locations, orientations, and positions. The atoms of our atomic decomposition contain both coarse-scale dyadic ``blobs,'' which are simply wedgelets from our dictionary, and fine-scale ``needles,'' which are differences of pairs of wedgelets.
The fine-scale atoms used in the adaptive atomic decomposition are highly anisotropic and occupy a range of positions, scales, and locations. This agrees qualitatively with the visual appearance of empirically determined sparse components of natural images. The set has certain definite scaling properties; for example, the number of atoms of length l scales as 1/l , and, when the object has α -smooth boundaries, the number of atoms with anisotropy \approx A scales as \approx A α-1 .
Similar content being viewed by others
Author information
Authors and Affiliations
Additional information
August 16, 1999. Date revised: April 24, 2000. Date accepted: April 4, 2000.
Rights and permissions
About this article
Cite this article
Donoho, D. Sparse Components of Images and Optimal Atomic Decompositions. Constr. Approx. 17, 353–382 (2001). https://doi.org/10.1007/s003650010032
Published:
Issue Date:
DOI: https://doi.org/10.1007/s003650010032
- Key words
- Sparse components analysis
- Independent components analysis
- Synthetic image models
- Computational harmonic analysis
- Edgelets
- Wedgelets