Skip to main content
SpringerLink
Log in

Shearlets: From Theory to Deep Learning

  • Living reference work entry
  • First Online:
Handbook of Mathematical Models and Algorithms in Computer Vision and Imaging

Abstract

Many important problem classes are governed by anisotropic features, which typically appear as singularities concentrated on lower-dimensional embedded manifolds. Examples include edges in images or shock fronts in solutions of transport-dominated equations. Shearlets are the first representation system which exhibits optimal sparse approximation properties in combination with a unified treatment of the continuum and digital realm, leading to faithful implementations. A prominent class of applications are inverse problems, foremost in imaging science, where shearlets are utilized for sparse regularization. Recently, shearlet systems have also been used in combination with data-driven approaches, predominately deep neural networks. This chapter shall serve as an introduction to and a survey about the theory of shearlets and their applications.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution

Institutional subscriptions

References

  • Adler, J., Öktem, O.: Solving ill-posed inverse problems using iterative deep neural networks. Inverse Probl. 33, 124007 (2017)

    Article  MathSciNet  MATH  Google Scholar 

  • Adler, J., Öktem, O.: Learned primal-dual reconstruction. IEEE T. Med. Imaging 37, 1322–1332 (2018)

    Article  Google Scholar 

  • Andrade-Loarca, H., Kutyniok, G., Öktem, O.: Shearlets as feature extractor for semantic edge detection: the model-based and data-driven realm. Proc. R. Soc. A. 476(2243), 20190841 (2020). https://royalsocietypublishing.org/toc/rspa/2020/476/2243

    Article  MathSciNet  Google Scholar 

  • Andrade-Loarca, H., Kutyniok, G., Öktem, O., Petersen, P.: Extraction of digital wavefront sets using applied harmonic analysis and deep neural networks. SIAM J. Imaging Sci. 12, 1936–1966 (2019)

    Article  MathSciNet  MATH  Google Scholar 

  • Antoine, J.P., Carrette, P., Murenzi, R., Piette, B.: Image analysis with two-dimensional continuous wavelet transform. Sig. Process. 31, 241–272 (1993)

    Article  MATH  Google Scholar 

  • Bamberger, R.H., Smith, M.J.T.: A filter bank for the directional decomposition of images: theory and design. IEEE Trans. Sig. Process. 40, 882–893 (1992)

    Article  Google Scholar 

  • Bodmann, B.G., Labate, D., Pahari, B.R.: Smooth projections and the construction of smooth Parseval frames of shearlets. Adv. Comput. Math. 45, 3241–3264 (2019)

    Article  MathSciNet  MATH  Google Scholar 

  • Bubba, T.A., Kutyniok, G., Lassas, M., März, M., Samek, W., Siltanen, S., Srinivasan, V.: Learning the invisible: a hybrid deep learning-shearlet framework for limited angle computed tomography. Inverse Probl. 35, 064002 (2019). https://iopscience.iop.org/article/10.1088/1361-6420/ab10ca

    Article  MathSciNet  MATH  Google Scholar 

  • Candès, E.J., Donoho, D.L.: New tight frames of curvelets and optimal representations of objects with piecewise C2 singularities. Commun. Pure Appl. Math. 56, 216–266 (2004)

    MATH  Google Scholar 

  • Canny, J.: A computational approach to edge detection. IEEE Trans. Pattern Anal. Mach. Intell. 8, 679–698 (1986)

    Article  Google Scholar 

  • Casazza, P.G., Kutyniok, G., Philipp, F.: Introduction to finite frame theory. In: Finite Frames: Theory and Applications, pp. 1–53. Birkhäuser, Boston (2012)

    Google Scholar 

  • Christensen, O.: An Introduction to Frames and Riesz Bases. Birkhäuser, Boston (2003)

    Book  MATH  Google Scholar 

  • Cohen, A.: Numerical Analysis of Wavelet Methods. Studies in Mathematics and Its Applications, vol. 32. JAI Press, Greenwich (2003)

    Google Scholar 

  • Dahlke, S., Kutyniok, G., Maass, P., Sagiv, C., Stark, H.-G., Teschke, G.: The uncertainty principle associated with the continuous shearlet transform. Int. J. Wavelets Multiresolut. Inf. Process. 6, 157–181 (2008)

    Article  MathSciNet  MATH  Google Scholar 

  • Dahlke, S., Kutyniok, G., Steidl, G., Teschke, G.: Shearlet coorbit spaces and associated banach frames. Appl. Comput. Harmon. Anal. 27, 195–214 (2009)

    Article  MathSciNet  MATH  Google Scholar 

  • Dahlke, S., Steidl, G., Teschke, G.: The continuous shearlet transform in arbitrary space dimensions. J. Fourier Anal. Appl. 16, 340–354 (2010)

    Article  MathSciNet  MATH  Google Scholar 

  • Dahlke, S., Steidl, G., Teschke, G.: Shearlet coorbit spaces: compactly supported analyzing shearlets, traces and embeddings. J. Fourier Anal. Appl. 17, 1232–1255 (2011)

    Article  MathSciNet  MATH  Google Scholar 

  • Dahlke, S., Häuser, S., Steidl, G., Teschke, G.: Shearlet coorbit spaces: traces and embeddings in higher dimensions. Monatsh. Math. 169, 15–32 (2013)

    Article  MathSciNet  MATH  Google Scholar 

  • Daubechies, I.: Ten Lectures on Wavelets. SIAM, Philadelphia (1992)

    Book  MATH  Google Scholar 

  • Daubechies, I., Defrise, M., De Mo, C.: An iterative thresholding algorithm for linear inverse problems with a sparsity constraint. Commun. Pure Appl. Math. 57, 1413–1457 (2004)

    Article  MathSciNet  MATH  Google Scholar 

  • Davenport, M., Duarte, M., Eldar, Y., Kutyniok, G.: Introduction to compressed sensing. In: Compressed Sensing: Theory and Applications, pp. 1–64. Cambridge University Press (2012)

    Google Scholar 

  • Do, M.N., Vetterli, M.: The contourlet transform: an efficient directional multiresolution image representation. IEEE Trans. Image Process. 14, 2091–2106 (2005)

    Article  Google Scholar 

  • Donoho, D.L.: Sparse components of images and optimal atomic decomposition. Constr. Approx. 17, 353–382 (2001)

    Article  MathSciNet  MATH  Google Scholar 

  • Donoho, D.L., Kutyniok, G.: Geometric separation using a wavelet-shearlet dictionary. SampTA’09, Marseille. Proceedings (2009)

    Google Scholar 

  • Donoho, D.L., Kutyniok, G.: Microlocal analysis of the geometric separation problem. Commun. Pure Appl. Math. 66, 1–47 (2013)

    Article  MathSciNet  MATH  Google Scholar 

  • Easley, G., Labate, D.: Image processing using shearlets. In: Shearlets: Multiscale Analysis for Multivariate Data, pp. 283–325. Birkhäuser, Boston (2012)

    Google Scholar 

  • Easley, G., Labate, D., Lim, W.-Q.: Sparse directional image representation using the discrete shearlet transform. Appl. Comput. Harmon. Anal. 25, 25–46 (2008)

    Article  MathSciNet  MATH  Google Scholar 

  • Genzel, M., Kutyniok, G.: Asymptotic analysis of inpainting via universal shearlet systems. SIAM J. Imaging Sci. 7, 2301–2339 (2014)

    Article  MathSciNet  MATH  Google Scholar 

  • Goodfellow, I., Bengio, Y., Courville, A.: Deep Learning. Adaptive Computation and Machine Learning. MIT Press, Cambridge (2017)

    MATH  Google Scholar 

  • Gottschling, N., Antun, V., Adcock, B., Hansen, A.C.: The troublesome kernel: why deep learning for inverse problems is typically unstable. preprint, arXiv:2001.01258 (2020)

    Google Scholar 

  • Gregor, K., LeCun, Y.: Learning fast approximations of sparse coding. In: International Conference on Machine Learning (ICML), pp. 399–406 (2010)

    Google Scholar 

  • Grohs, P.: Continuous Shearlet frames and Resolution of the Wavefront Set. Monatsh. Math. 164, 393–426 (2011a)

    Article  MathSciNet  MATH  Google Scholar 

  • Grohs, P.: Continuous shearlet tight frames. J. Fourier Anal. Appl. 17, 506–518 (2011b)

    Article  MathSciNet  MATH  Google Scholar 

  • Grohs, P.: Bandlimited shearlet frames with nice duals. J. Comput. Appl. Math. 142, 139–151 (2013)

    Article  MathSciNet  MATH  Google Scholar 

  • Grohs, P., Kutyniok, G.: Parabolic molecules. Found. Comput. Math. 14, 299–337 (2014)

    Article  MathSciNet  MATH  Google Scholar 

  • Grohs, P., Keiper, S., Kutyniok, G., Schäfer, M.: α-molecules. Appl. Comput. Harmon. Anal. 42, 297–336 (2016a)

    Google Scholar 

  • Grohs, P., Keiper, S., Kutyniok, G., Schäfer, M.: Cartoon approximation with α-curvelets. J. Fourier Anal. Appl. 22, 1235–1293 (2016b)

    Article  MathSciNet  MATH  Google Scholar 

  • Gu, J., Ye, J.C.: Multi-scale wavelet domain residual learning for limited-angle CT reconstruction. In: Procs Fully3D, pp. 443–447 (2017)

    Google Scholar 

  • Guo, K., Labate, D.: Optimally sparse multidimensional representation using shearlets. SIAM J Math. Anal. 39, 298–318 (2007)

    Article  MathSciNet  MATH  Google Scholar 

  • Guo, K., Labate, D.: The construction of smooth parseval frames of shearlets. Math. Model. Nat. Phenom. 8, 82–105 (2013)

    Article  MathSciNet  MATH  Google Scholar 

  • Guo, K., Kutyniok, G., Labate, D.: Sparse multidimensional representations using anisotropic dilation and shear operators. In: Wavelets and Splines, Athens, 2005, pp. 189–201. Nashboro Press, Nashville (2006)

    Google Scholar 

  • Guo, K., Labate, D., Lim, W.-Q.: Edge analysis and identification using the continuous shearlet transform. Appl. Comput. Harmon. Anal. 27, 24–46 (2009)

    Article  MathSciNet  MATH  Google Scholar 

  • Häuser, S., Steidl, G.: Convex multiclass segmentation with shearlet regularization. Int. J. Comput. Math. 90, 62–81 (2013)

    Article  MathSciNet  MATH  Google Scholar 

  • Hörmander, L.: The analysis of linear partial differential operators. I. Distribution theory and Fourier analysis. Springer, Berlin (2003)

    Book  MATH  Google Scholar 

  • Jin, K.H., McCann, M.T., Froustey, E., Unser, M.: Deep convolutional neural network for inverse problems in imaging. IEEE Trans. Image Proc. 26, 4509–4522 (2017)

    Article  MathSciNet  MATH  Google Scholar 

  • King, E.J., Kutyniok, G., Zhuang, X.: Analysis of inpainting via clustered sparsity and microlocal analysis. J. Math. Imaging Vis. 48, 205–234 (2014)

    Article  MathSciNet  MATH  Google Scholar 

  • Kittipoom, P., Kutyniok, G., Lim, W.-Q.: Irregular shearlet frames: geometry and approximation properties. J. Fourier Anal. Appl. 17, 604–639 (2011)

    Article  MathSciNet  MATH  Google Scholar 

  • Kittipoom, P., Kutyniok, G., Lim, W.-Q.: Construction of compactly supported shearlet frames. Constr. Approx. 35, 21–72 (2012)

    Article  MathSciNet  MATH  Google Scholar 

  • Kutyniok, G.: Clustered sparsity and separation of cartoon and texture. SIAM J. Imaging Sci. 6, 848–874 (2013)

    Article  MathSciNet  MATH  Google Scholar 

  • Kutyniok, G.: Geometric separation by single-pass alternating thresholding. Appl. Comput. Harmon. Anal. 36, 23–50 (2014)

    Article  MathSciNet  MATH  Google Scholar 

  • Kutyniok, G., Labate, D.: Resolution of the wavefront set using continuous shearlets. Trans. Am. Math. Soc. 361, 2719–2754 (2009)

    Article  MathSciNet  MATH  Google Scholar 

  • Kutyniok, G., Labate, D.: Introduction to shearlets. In: Shearlets: Multiscale Analysis for Multivariate Data, pp. 1–3. Birkhäuser, Boston (2012)

    Google Scholar 

  • Kutyniok, G., Lim, W.-Q.: Compactly supported shearlets are optimally sparse. J. Approx. Theory 163, 1564–1589 (2011)

    Article  MathSciNet  MATH  Google Scholar 

  • Kutyniok, G., Lim, W.-Q.: Image separation using wavelets and shearlets. In: Curves and Surfaces, Avignon, 2010). Lecture Notes in Computer Science, vol. 6920, pp. 416–430. Springer (2012)

    Google Scholar 

  • Kutyniok, G., Lim, W.-Q.: Optimal compressive imaging of Fourier data. SIAM J. Imaging Sci. 11, 507–546 (2018)

    Article  MathSciNet  MATH  Google Scholar 

  • Kutyniok, G., Petersen, P.: Classification of edges using compactly supported shearlets. Appl. Comput. Harmon. Anal. 42, 245–293 (2017)

    Article  MathSciNet  MATH  Google Scholar 

  • Kutyniok, G., Lemvig, J., Lim, W.-Q.: Optimally sparse approximations of 3D functions by compactly supported shearlet frames. SIAM J. Math. Anal. 44, 2962–3017 (2012)

    Article  MathSciNet  MATH  Google Scholar 

  • Kutyniok, G., Lim, W.-Q., Reisenhofer, R.: ShearLab 3D: faithful digital shearlet transforms based on compactly supported shearlets. ACM Trans. Math. Softw. 42, 5 (2016)

    Article  MathSciNet  MATH  Google Scholar 

  • Labate, D., Lim, W.-Q., Kutyniok, G., Weiss, G.: Sparse multidimensional representation using shearlets. In: Wavelets XI, Proceedings of SPIE, Bellingham, vol. 5914, pp. 254–262 (2005)

    Google Scholar 

  • Labate, D., Mantovani, L., Negi, P.S.: Shearlet smoothness spaces. J. Fourier Anal. Appl. 19, 577–611 (2013)

    Article  MathSciNet  MATH  Google Scholar 

  • Le Pennec, E.L., Mallat, S.: Sparse geometric image representations with bandelets. IEEE Trans. Image Process. 14, 423–438 (2005)

    Article  MathSciNet  Google Scholar 

  • Lessig, C., Petersen, P., Schäfer, M.: Bendlets: a second-order shearlet transform with bent elements. Appl. Comput. Harmon. Anal. 46, 384–399 (2019)

    Article  MathSciNet  MATH  Google Scholar 

  • Lim, W.-Q.: The discrete shearlet transform: a new directional transform and compactly supported shearlet frames. IEEE Trans. Image Proc. 19, 1166–1180 (2010)

    Article  MathSciNet  MATH  Google Scholar 

  • Lim, W.-Q.: Nonseparable shearlet transform. IEEE Trans. Image Proc. 22, 2056–2065 (2013)

    Article  MathSciNet  MATH  Google Scholar 

  • Mallat, S.: A Wavelet Tour of Signal Processing. Academic Press, San Diego (1998)

    MATH  Google Scholar 

  • McCann, M.T., Jin, K.H., Unser, M.: Convolutional neural networks for inverse problems in imaging: a review. IEEE Signal Proc. Mag. 34, 85–95 (2017)

    Article  Google Scholar 

  • Meinhardt, T., Möller, M., Hazirbas, C., Cremers, D.: Learning proximal operators: using denoising networks for regularizing inverse imaging problems. In: International Conference on Computer Vision (ICCV) (2017)

    Google Scholar 

  • Natterer, F.: The Mathematics of Computerized Tomography. Society for Industrial and Applied Mathematics (SIAM), Philadelphia (2001)

    Google Scholar 

  • Quinto, E.T.: Singularities of the X-ray transform and limited data tomography in \(\mathbb {R}^2\) and \(\mathbb {R}^3\). SIAM J. Math. Anal. 24, 1215–1225 (1993)

    Google Scholar 

  • Reisenhofer, R., Kiefer, J., King, E.J.: Shearlet-based detection of flame fronts. Exp. Fluids 57, 11 (2015)

    Google Scholar 

  • Reisenhofer, R., Bosse, S., Kutyniok, G., Wiegand, T.: A Haar wavelet-based perceptual similarity index for image quality assessment. Sig. Process. Image 61, 33–43 (2018)

    Article  Google Scholar 

  • Ronneberger, O., Fischer, P., Brox, T.: U-Net: convolutional networks for biomedical image segmentation. In: Medical Image Computing and Computer-Assisted Intervention (MICCAI). LNCS, vol. 9351, pp. 234–241. Springer (2015)

    Google Scholar 

  • Simoncelli, E.P., Freeman, W.T., Adelson, E.H., Heeger, D.J.: Shiftable multiscale transforms. IEEE Trans. Inform. Theory 38, 587–607 (1992)

    Article  MathSciNet  Google Scholar 

  • Starck, J.-L., Murtagh, F., Fadili, J.: Sparse Image and Signal Processing: Wavelets, Curvelets, Morphological Diversity. Cambridge University Press, Cambridge (2010)

    Book  MATH  Google Scholar 

  • Yi, S., Labate, D., Easley, G.R., Krim, H.: A shearlet approach to edge analysis and detection. IEEE Trans. Image Process. 18, 929–941 (2009)

    Article  MathSciNet  MATH  Google Scholar 

Download references

Acknowledgements

G.K. would like to thank Hector Andrade-Loarca for producing several of the figures.

Author information

Authors and Affiliations

  1. Ludwig-Maximilians-Universität München, Mathematisches Institut, München, Germany

    Gitta Kutyniok

Authors
  1. Gitta Kutyniok
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Gitta Kutyniok .

Editor information

Editors and Affiliations

  1. Department of Mathematical Sciences, The University of Liverpool, Liverpool, UK

    Ke Chen

  2. Dept Appl Math & Theoretical Phys, University of Cambridge, Cambridge, UK

    Carola-Bibiane Schönlieb

  3. Department of Mathematics, Hong Kong Baptist University, Kowloon Tong, Hong Kong

    Xue-Cheng Tai

  4. Department of Applied Mathematics & Stat, Johns Hopkins University, Baltimore, MD, USA

    Laurent Younces

Section Editor information

  1. No affiliation provided

    Ke Chen Ph.D

  2. No affiliation provided

    Xue-Cheng Tai Ph.D

  3. No affiliation provided

    Carola-Bibiane Schönlieb Ph.D

  4. No affiliation provided

    Laurent Younes

Rights and permissions

Reprints and permissions

Copyright information

© 2021 Springer Nature Switzerland AG

About this entry

Check for updates. Verify currency and authenticity via CrossMark

Cite this entry

Kutyniok, G. (2021). Shearlets: From Theory to Deep Learning. In: Chen, K., Schönlieb, CB., Tai, XC., Younces, L. (eds) Handbook of Mathematical Models and Algorithms in Computer Vision and Imaging. Springer, Cham. https://doi.org/10.1007/978-3-030-03009-4_80-1

Download citation

  • DOI: https://doi.org/10.1007/978-3-030-03009-4_80-1

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-030-03009-4

  • Online ISBN: 978-3-030-03009-4

  • eBook Packages: Springer Reference MathematicsReference Module Computer Science and Engineering

Publish with us

Policies and ethics

Search

Navigation