Skip to main content
Log in

Discrete and Continuous Models for Partitioning Problems

  • Published:
International Journal of Computer Vision Aims and scope Submit manuscript

Abstract

Recently, variational relaxation techniques for approximating solutions of partitioning problems on continuous image domains have received considerable attention, since they introduce significantly less artifacts than established graph cut-based techniques. This work is concerned with the sources of such artifacts. We discuss the importance of differentiating between artifacts caused by discretization and those caused by relaxation and provide supporting numerical examples. Moreover, we consider in depth the consequences of a recent theoretical result concerning the optimality of solutions obtained using a particular relaxation method. Since the employed regularizer is quite tight, the considered relaxation generally involves a large computational cost. We propose a method to significantly reduce these costs in a fully automatic way for a large class of metrics including tree metrics, thus generalizing a method recently proposed by Strekalovskiy and Cremers (IEEE conference on computer vision and pattern recognition, pp. 1905–1911, 2011).

This is a preview of subscription content, log in via an institution to check access.

Access this article

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11
Fig. 12
Fig. 13
Fig. 14
Fig. 15
Fig. 16
Fig. 17
Fig. 18
Fig. 19
Fig. 20
Fig. 21
Fig. 22
Fig. 23

Similar content being viewed by others

References

  • Ambrosio, L., Fusco, N., & Pallara, D. (2000). Functions of bounded variation and free discontinuity problems. Oxford: Clarendon Press.

    MATH  Google Scholar 

  • Appleton, B., & Talbot, H. (2006). Globally minimal surfaces by continuous maximal flows. IEEE Transactions on Pattern Analysis and Machine Intelligence, 28, 106–118.

    Article  Google Scholar 

  • Bae, E., Yuan, J., & Tai, X. C. (2011). Global minimization for continuous multiphase partitioning problems using a dual approach. International Journal of Computer Vision, 92, 112–129.

    Article  MathSciNet  MATH  Google Scholar 

  • Bartal, Y. (1998). On approximating arbitrary metrices by tree metrics. In ACM symposium on theory of computing.

  • Bertsekas, D. P. (1998). Network optimization: Continuous and discrete models. Belmont, MA: Athena Scientific.

    MATH  Google Scholar 

  • Blake, A., & Zisserman, A. (1987). Visual reconstruction. Cambridge: MIT Press.

    Google Scholar 

  • Boykov, Y. (2003). Computing geodesics and minimal surfaces via graph cuts. In The international conference on computer vision (pp. 26–33).

  • Boykov, Y., Veksler, O., & Zabih, R. (2001). Fast approximate energy minimization via graph cuts. IEEE Transactions on Pattern Analysis and Machine Intelligence, 23(11), 1222–1239.

    Article  Google Scholar 

  • Braides, A. (2002). Gamma-convergence for beginners. Oxford: Oxford University Press.

    Book  MATH  Google Scholar 

  • Călinescu, G., Karloff, H., & Rabani, Y. (2000). An improved approximation algorithm for multiway cut. Journal of Computer and System Sciences, 60, 564–574.

    Article  MathSciNet  MATH  Google Scholar 

  • Caselles, V., Kimmel, R., & Sapiro, G. (1997). Geodesic active contours. International Journal of Computer Vision, 22, 61–79.

    Article  MATH  Google Scholar 

  • Chambolle, A. (1999). Finite-differences discretizations of the Mumford–Shah functional. Modélisation Mathématique et Analyse Numérique, 33, 261–288.

    Article  MathSciNet  MATH  Google Scholar 

  • Chambolle, A., & Darbon, J. (2009). On total variation minimization and surface evolution using parametric maximum flows. International Journal of Computer Vision, 84, 288–307.

    Article  Google Scholar 

  • Chambolle, A., Levine, S. E., & Lucier, B. J. (2011). An upwind finite-difference method for total variation-based image smoothing. Journal of Imaging Science, 4(1), 277–299.

    Article  MathSciNet  MATH  Google Scholar 

  • Chan, T. F., Esedoglu, S., & Nikolova, M. (2006). Algorithms for finding global minimizers of image segmentation and denoising models. Journal of Applied Mathematics, 66(5), 1632–1648.

    MathSciNet  MATH  Google Scholar 

  • Chan, T. F., & Vese, L. A. (2001). Active contours without edges. IEEE Transactions on Image Processing, 10(2), 266–277.

    Article  MATH  Google Scholar 

  • Charikar, M., Chekuri, C., Goel, A., Guha, S., & Plotkin, S. (1998). Approximating a finite metric by a small number of tree metrics. In ACM Foundations on Computer Science (pp. 379–388).

  • Combettes, P. L., & Pesquet, J. C. (2010). Proximal splitting methods in signal processing. In Fixed-point algorithms for inverse problems in science and engineering. New York: Springer.

  • Couprie, C., Grady, L., Najman, L., & Talbot, H. (2011). Power watersheds: A unifying graph-based optimization framework. IEEE Transactions on Pattern Analysis and Machine Intelligence, 33(7), 1384–1399.

    Article  Google Scholar 

  • Couprie, C., Grady, L., Talbot, H., & Najman, L. (2011). Combinatorial continuous maximum flow. Journal of Imaging Science, 4(3), 905–930.

    Article  MathSciNet  MATH  Google Scholar 

  • Dahlhaus, E., Johnson, D. S., Papadimitriou, C. H., & Seymour, P. D. (1994). The complexity of multiterminal cuts. Journal of Computing, 23(4), 864–894.

    MathSciNet  MATH  Google Scholar 

  • Dal Maso G. (1993). An introduction to \(\Gamma \)-convergence. Basel: Birkhäuser.

  • De Giorgi, E., & Franzoni, T. (1975). Su un tipo di convergenza variazionale. Atti Accad. Naz. Lincei Rend Cl. Sci. Fis. Mat. Natur., 68, 842–850.

    Google Scholar 

  • Dixit, N., Keriven, R., & Paragios, N. (2005). GPU-cuts: Combinatorial optimisation, graphic processing units and adaptive object extraction. Tech. Rep. 05–07, CERTIS, ENPC.

  • Esser, E., Zhang, X., & Chan, T. F. (2010). A general framework for a class of first order primal-dual algorithms for convex optimization in imaging science. Journal of Imaging Science, 3, 1015–1046.

    Google Scholar 

  • Fakcharoenphol, J., Rao, S., & Talwar, K. (2004). A tight bound on approximating arbitrary metrics by tree metrics. Journal of Computer and System Sciences, 69(3), 485–497.

    Article  MathSciNet  MATH  Google Scholar 

  • Faugeras, O., & Luong, Q. T. (2001). The geometry of multiple images. Cambridge: MIT Press.

    MATH  Google Scholar 

  • Geman, S., & Geman, D. (1984). Stochastic relaxation, Gibbs distributions and the Bayesian restoration of images. IEEE Transactions on Pattern Analysis and Machine Intelligence, 6, 721–741.

    Article  MATH  Google Scholar 

  • Gobbino, M., & Mora, M. G. (2001). Finite-difference approximation of free-discontinuity problems. Proceedings of the Royal Society of Edinburgh, 131, 567–595.

    Google Scholar 

  • Goldluecke, S., & Cremers, D. (2010). Convex relaxation for multilabel problems with product label spaces. In European conference of computer vision (pp. 225–238).

  • Goldschlager, L. M., Shaw, R. A., & Staples, J. (1982). The maximum flow problem is log space complete for P. Theoretical Computer Science, 21, 105–111.

    Article  MathSciNet  MATH  Google Scholar 

  • Grady, L. J., & Polimeni, J. R. (2010). Discrete calculus: Applied analysis on graphs for computational science. New York: Springer.

    Book  Google Scholar 

  • Hartley, R., & Zisserman, A. (2000). Multiple view geometry in computer vision. Cambridge: Cambridge University Press.

    MATH  Google Scholar 

  • Ising, E. (1925). Beitrag zur Theorie des Ferromagnetismus. Zeitschrift fur Physik, 31(1), 253–258.

    Article  Google Scholar 

  • Kass, M., Witkin, A., & Terzopoulos, D. (1987). Snakes: Active contours models. International Journal of Computer Vision, 1(4), 321–331.

    Article  Google Scholar 

  • Kleinberg, J. M., & Tardos, E. (1999). Approximation algorithms for classification problems with pairwise relationships: Metric labeling and Markov random fields. In Foundations of computer science (pp. 14–23).

  • Klodt, M., Schoenemann, T., Kolev, K., Schikora, M., & Cremers, D. (2008). An experimental comparison of discrete and continuous shape optimization methods. In European conference of computer vision (pp. 332–345). France: Marseille.

  • Koller, D., & Friedman, N. (2009). Probabilistic graphical models: Principles and techniques. Cambridge, MA: MIT Press.

    Google Scholar 

  • Kolmogorov, V., & Boykov, Y. (2005). What metrics can be approximated by geo-cuts, or global optimization of length/area and flux. International conference on computer vision (Vol. 1, pp. 564–571).

  • Komodakis, N., & Tziritas, G. (2007). Approximate labeling via graph cuts based on linear programming. IEEE Transactions on Pattern Analysis and Machine Intelligence, 29(8), 1436–1453.

    Article  Google Scholar 

  • Lauritzen, S. (1996). Graphical models. Oxford: Oxford University Press.

    Google Scholar 

  • Lellmann, J. (2011). Nonsmooth convex variational approaches to image analysis. Ph.D. thesis, University of Heidelberg.

  • Lellmann, J., Breitenreicher, D., & Schnörr, C. (2010). Fast and exact primal-dual iterations for variational problems in computer vision. In European conference on computer vision. LNCS (vol. 6312, pp. 494–505). Berlin: Springer.

  • Lellmann, J., Kappes, J., Yuan, J., Becker, F., & Schnörr, C. (2009). Convex multi-class image labeling by simplex-constrained total variation. In Scale space and variable methods. LNCS (Vol. 5567, pp. 150–162). Berlin: Springer.

  • Lellmann, J., Lenzen, F., & Schnörr, C. (2011). Optimality bounds for a variational relaxation of the image partitioning problem. In Energy minimization methods and applications in computer vision and pattern recognition.

  • Lellmann, J., Lenzen, F., & Schnörr, C. (2012). Optimality bounds for a variational relaxation of the image partitioning problem. Journal of Mathematical Imaging and Vision. doi:10.1007/s10851-012-0390-7.

  • Lellmann, J., & Schnörr, C. (2011). Continuous multiclass labeling approaches and algorithms. Journal of Imaging Science, 4(4), 1049–1096.

    Google Scholar 

  • Lempitsky, V., Rother, C., & Blake, A. (2007). Logcut—Efficient graph cut optimization for Markov random fields. In European conference on computer vision (pp. 1–8).

  • Lempitsky, V., Rother, C., Roth, S., & Blake, A. (2010). Fusion moves for Markov random field optimization. IEEE Transactions on Pattern Analysis and Machine Intelligence, 32(8), 1392–1405.

    Google Scholar 

  • Lie, J., Lysaker, M., & Tai, X. C. (2006). A variant of the level set method and applications to image segmentation. Mathematics of Computation, 75, 1155–1174.

    Google Scholar 

  • Morel, J. M., & Solimini, S. (1995). Variational methods in image segmentation. Basel: Birkhäuser.

    Book  Google Scholar 

  • Mumford, D., & Shah, J. (1989). Optimal approximations by piecewise smooth functions and associated variational problems. Communications on Pure and Applied Mathematics, 42, 577–685.

    Article  MathSciNet  MATH  Google Scholar 

  • Olsson, C. (2009). Global optimization in computer vision: Convexity, cuts and approximation algorithms. Ph.D. thesis, Lund University.

  • Olsson, C., Byröd, M., Overgaard, N. C., & Kahl, F. (2009). Extending continuous cuts: Anisotropic metrics and expansion moves. In International conference on computer vision.

  • Paragios, N., Chen, Y., & Faugeras, O. (Eds.). (2006). The handbook of mathematical models in computer vision. New York: Springer.

  • Pock, T., Chambolle, A., Cremers, D., & Bischof, H. (2009). A convex relaxation approach for computing minimal partitions. In Computer vision on pattern recognition (pp. 810–817).

  • Pock, T., Schönemann, T., Graber, G., Bischof, H., & Cremers, D. (2008). A convex formulation of continuous multi-label problems. In European conference on computer vision (Vol. 3, pp. 792–805).

  • Rickett, J., & Fomel, S. (1999). A second-order fast marching Eikonal solver. Tech. Rep. 100, Standford Exploration Project Report.

  • Scherzer, O., Grasmair, M., Grossauer, H., Haltmeier, M., & Lenzen, F. (2009). Variational Methods in Imaging. Applied Mathematical Sciences (Vol. 167). Berlin: Springer.

  • Sinop, A. K., & Grady, L. (2007). A seeded image segmentation framework unifying graph cuts and random walker which yields a new algorithm. In International conference on computer vision (pp. 1–8).

  • Strang, G. (1983). Maximal flow through a domain. Mathematical Programming, 26, 123–143.

    Article  MathSciNet  MATH  Google Scholar 

  • Strekalovskiy, E., & Cremers, D. (2011). Total variation for cyclic structures: Convex relaxation and efficient minimization. In Conference on computer vision and pattern recognition (pp. 1905–1911).

  • Strekalovskiy, E., Goldluecke, B., & Cremers, D. (2011). Tight convex relaxations for vector-valued labeling problems. In International conference on computer vision (pp. 2328–2335).

  • Trobin, W., Pock, T., Cremers, D., & Bischof, H. (2008). Continuous energy minimization by repeated binary fusion. In European conference on computer vision (Vol. 4, pp. 667–690).

  • Zach, C., Gallup, D., Frahm, J. M., & Niethammer, M. (2008). Fast global labeling for real-time stereo using multiple plane sweeps. In Vision, modeling, and visualization.

Download references

Acknowledgments

The second and third author were supported by Engineering and Physical Sciences Research Council (EPSRC)-Project EP/H016317/1. This publication is partly based on work supported by Award No. KUK-I1- 007-43, made by King Abdullah University of Science and Technology (KAUST).

Author information

Authors and Affiliations

Authors

Corresponding author

Correspondence to Jan Lellmann.

Appendix

Appendix

Proof

(Proposition 3) Let \(v \in \mathcal D _{\mathrm{loc}}^m\) and \(i \ne j\). Denote a shortest path in \(\mathcal G ^m\) from \(i\) to \(j\) by \((k_1=i,k_2,\ldots ,k_{n-1},k_n=j)\) for some \(n \geqslant 2\). Then, due to the triangle inequality of the norm,

$$\begin{aligned} \Vert v^i - v^j\Vert _2 \leqslant \Vert v^{k_1} - v^{k_2} \Vert _2 + \ldots + \Vert v^{k_{n-1}} - v^{k_n}\Vert _2. \end{aligned}$$
(124)

Since \(v \in \mathcal D _{\mathrm{loc}}^m\), the terms on the right can be bounded,

$$\begin{aligned} \Vert v^i - v^j\Vert _2 \leqslant m(k_1,k_2) + \ldots +m(k_{n-1},k_n). \end{aligned}$$
(125)

By definition, the sum equals \(\bar{m}(i,j)\) and therefore by assumption \(\bar{m^{\prime }}(i,j)\), thus

$$\begin{aligned} \Vert v^i - v^j\Vert _2 \leqslant \bar{m}(i,j)=\bar{m^{\prime }}(i,j) \leqslant m^{\prime }(i,j). \end{aligned}$$
(126)

Consequently \(v \in \mathcal D _{\mathrm{loc}}^{m^{\prime }}\), and, since \(v\) was arbitrary, \(\mathcal D _{\mathrm{loc}}^m \subseteq \mathcal D _{\mathrm{loc}}^{m^{\prime }}\). By symmetry the reverse inclusion holds as well, and therefore equality. \(\square \)

Proof

(Proposition 4) Since \(d\) is a metric and thus satisfies the triangle inequality, (114) is equivalent to

$$\begin{aligned} d(i,k) + d(k,j) = d(i,j). \end{aligned}$$
(127)

Since \(d\) is the shortest-path metric of \(m\), Eq. (127) states that there exists a shortest path from \(i\) to \(j\) in \(\mathcal G ^m\) passing through \(k\). This path consists of at least two edges which both have positive weight by the definition of \(\mathcal M \). Therefore the path cannot contain the edge \((i,j)\), since \(m(i,j) \geqslant d(i,j)\) which would contradict (127).

This means that the path is also contained in \(\mathcal G ^{m^{\prime }}\). By construction of \(m^{\prime }\) it must then also be a shortest path from \(i\) to \(j\) in \(m^{\prime }\), which shows that \(\bar{m^{\prime }}(i,j) = \bar{m}(i,j)\). Since \(m^{\prime }(i^{\prime },j^{\prime }) = m(i^{\prime },j^{\prime })\) for all \((i^{\prime },j^{\prime }) \ne (i,j)\), this immediately implies \(\bar{m^{\prime }} = \bar{m}\) and therefore the assertion. \(\square \)

Proof

(Proposition 5) Define

$$\begin{aligned} m(i,j):=\left\{ \begin{array}{ll} +\infty , &{} \exists k\ne i,j : d(i,j) \geqslant d(i,k) + d(k,j)\\ d(i,j), &{} \text{ otherwise }. \end{array}\right. \end{aligned}$$

An iterative application of Proposition 4 shows that \(\bar{m} = d\). Proposition 3 then states that their associated dual constraint sets coincide,

$$\begin{aligned} \mathcal D _{\mathrm{loc}}^m = \mathcal D _{\mathrm{loc}}^d. \end{aligned}$$
(128)

Since \(\bigcap _{(i,j) \in \mathcal R } \mathcal D _{i j} = \mathcal D _{\mathrm{loc}}^m\) by construction, this concludes the proof. \(\square \)

Rights and permissions

Reprints and permissions

About this article

Cite this article

Lellmann, J., Lellmann, B., Widmann, F. et al. Discrete and Continuous Models for Partitioning Problems. Int J Comput Vis 104, 241–269 (2013). https://doi.org/10.1007/s11263-013-0621-4

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11263-013-0621-4

Keywords

Navigation