Skip to main content
SpringerLink
Log in

Cluster Sampling Methods

  • Chapter
  • First Online:
Monte Carlo Methods

Abstract

The Swendsen-Wang algorithm has been initially designed for addressing the critical slowing down in sampling the Ising and Potts models described below at or near the critical temperature where phase transitions occur. Fortuin and Kasteleyn [17] have mapped the Potts model to a percolation model [7]. The percolation model is a model for a porous material with randomly distributed pores though which a liquid can percolate. The model is defined on a set of nodes (e.g. organized on a lattice), each node having a label sampled independently from a Bernoulli random variable with expectation p, where label 1 represents a pore. Two adjacent nodes with both having label 1 are automatically connected by an edge. This way random clusters of nodes are obtained by sampling the node labels and automatically connecting adjacent nodes that have label 1.

“Shapes that contain no inner components of positive/negative relationships will function better with shapes of the same nature” – Keith Haring

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 89.00
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Hardcover Book
USD 119.99
Price excludes VAT (USA)
  • Durable hardcover edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

Notes

  1. 1.

    http://lmb.informatik.uni-freiburg.de/resources/datasets/

References

  1. Ackermann W (1928) Zum hilbertschen aufbau der reellen zahlen. Math Ann 99(1):118–133

    Article  MathSciNet  MATH  Google Scholar 

  2. Apt KR (1999) The essence of constraint propagation. Theor Comput Sci 221(1):179–210

    Article  MathSciNet  MATH  Google Scholar 

  3. Barbu A, Zhu S-C (2005) Generalizing Swendsen-Wang to sampling arbitrary posterior probabilities. IEEE Trans Pattern Anal Mach Intell 27(8):1239–1253

    Article  Google Scholar 

  4. Barbu A, Zhu S-C (2007) Generalizing Swendsen-Wang for image analysis. J Comput Graph Stat 16(4):877

    Article  MathSciNet  Google Scholar 

  5. Besag J (1986) On the statistical analysis of dirty pictures. J R Stat Soc Ser B (Methodol) 48(3):259–302

    MathSciNet  MATH  Google Scholar 

  6. Boykov Y, Veksler O, Zabih R (2001) Fast approximate energy minimization via graph cuts. IEEE Trans Pattern Anal Mach Intell 23(11):1222–1239

    Article  Google Scholar 

  7. Broadbent SR, Hammersley JM (1957) Percolation processes: I. crystals and mazes. In: Mathematical proceedings of the Cambridge philosophical society, vol 53. Cambridge University Press, pp 629–641

    Google Scholar 

  8. Brox T, Malik J (2010) Object segmentation by long term analysis of point trajectories. In: ECCV, pp 282–295

    Google Scholar 

  9. Chui H, Rangarajan A (2003) A new point matching algorithm for non-rigid registration. Comput Vis Image Underst 89(2):114–141

    Article  MATH  Google Scholar 

  10. Cooper C, Frieze AM (1999) Mixing properties of the Swendsen-Wang process on classes of graphs.Random Struct Algor 15(3–4):242–261

    Article  MATH  Google Scholar 

  11. Cormen TH, Leiserson CE, Rivest RL, Stein C, et al (2001) Introduction to algorithms, vol 2. MIT Press, Cambridge

    MATH  Google Scholar 

  12. Ding L, Barbu A, Meyer-Baese A (2012) Motion segmentation by velocity clustering with estimation of subspace dimension. In: ACCV workshop on detection and tracking in challenging environments

    Google Scholar 

  13. Ding L, Barbu A (2015) Scalable subspace clustering with application to motion segmentation. In: Current trends in Bayesian methodology with applications. CRC Press, Boca Raton, p 267

    Chapter  Google Scholar 

  14. Edwards RG, Sokal AD (1988) Generalization of the Fortuin-Kasteleyn-Swendsen-Wang representation and monte carlo algorithm. Phys Rev D 38(6):2009

    Article  MathSciNet  Google Scholar 

  15. Elhamifar E, Vidal R (2009) Sparse subspace clustering. In: CVPR

    Book  MATH  Google Scholar 

  16. Felzenszwalb PF, Schwartz JD (2007) Hierarchical matching of deformable shapes. In: CVPR, pp 1–8

    Google Scholar 

  17. Fortuin CM, Kasteleyn PW (1972) On the random-cluster model: I. introduction and relation to other models. Physica 57(4):536–564

    Article  MathSciNet  Google Scholar 

  18. Fredman M, Saks M (1989) The cell probe complexity of dynamic data structures. In: Proceedings of the twenty-first annual ACM symposium on theory of computing, pp 345–354

    Google Scholar 

  19. Galler BA, Fisher MJ (1964) An improved equivalence algorithm. Commun ACM 7(5):301–303

    Article  MATH  Google Scholar 

  20. Gelman A, Carlin JB, Stern HS, Dunson DB, Vehtari A, Rubin DB (2013) Bayesian data analysis. CRC Press, Boca Raton/London/New York

    MATH  Google Scholar 

  21. Geman S, Geman D (1984) Stochastic relaxation, gibbs distributions, and the Bayesian restoration of images. IEEE Trans Pattern Anal Mach Intell 6:721–741

    Article  MATH  Google Scholar 

  22. Gilks WR, Roberts GO (1996) Strategies for improving MCMC. In: Markov chain Monte Carlo in practice. Springer, Boston, pp 89–114

    Google Scholar 

  23. Gore VK, Jerrum MR (1999) The Swendsen–Wang process does not always mix rapidly. J Stat Phys 97(1–2):67–86

    Article  MathSciNet  MATH  Google Scholar 

  24. Hastings WK (1970) Monte carlo sampling methods using Markov chains and their applications. Biometrika 57(1):97–109

    Article  MathSciNet  MATH  Google Scholar 

  25. Higdon DM (1998) Auxiliary variable methods for Markov chain monte carlo with applications. J Am Stat Assoc 93(442):585–595

    Article  MATH  Google Scholar 

  26. Huber M (2003) A bounding chain for Swendsen-Wang. Random Struct Algor 22(1):43–59

    Article  MathSciNet  MATH  Google Scholar 

  27. Kirkpatrick S, Vecchi MP, et al (1983) Optimization by simulated annealing. Science 220(4598):671–680

    Article  MathSciNet  MATH  Google Scholar 

  28. Kolmogorov V, Rother C (2007) Minimizing nonsubmodular functions with graph cuts-a review. IEEE Trans Pattern Anal Mach Intell 29(7):1274–1279

    Article  Google Scholar 

  29. Kumar MP, Torr PHS (2006) Fast memory-efficient generalized belief propagation. In: Computer vision–ECCV 2006. Springer, pp 451–463

    Google Scholar 

  30. Kumar S, Hebert M (2003) Man-made structure detection in natural images using a causal multiscale random field. In: CVPR, vol 1. IEEE, pp I–119

    Google Scholar 

  31. Lafferty JD, McCallum A, Pereira FCN (2001) Conditional random fields: probabilistic models for segmenting and labeling sequence data. In: Proceedings of the eighteenth international conference on machine learning. Morgan Kaufmann Publishers Inc., pp 282–289

    Google Scholar 

  32. Lauer F, Schnörr C (2009) Spectral clustering of linear subspaces for motion segmentation. In: ICCV

    Book  Google Scholar 

  33. Lin L, Zeng K, Liu X, Zhu S-C (2009) Layered graph matching by composite cluster sampling with collaborative and competitive interactions. In: CVPR, pp 1351–1358

    Google Scholar 

  34. Liu G, Lin Z, Yu Y (2010) Robust subspace segmentation by low-rank representation. In: ICML

    Google Scholar 

  35. Liu JS, Wong WH, Kong A (1995) Covariance structure and convergence rate of the gibbs sampler with various scans. J R Stat Soc Ser B (Methodol) 57(1):157–169

    MathSciNet  MATH  Google Scholar 

  36. Liu JS, Wu YN (1999) Parameter expansion for data augmentation. J Am Stat Assoc 94(448):1264–1274

    Article  MathSciNet  MATH  Google Scholar 

  37. Mackworth AK (1977) Consistency in networks of relations. Artif Intell 8(1):99–118

    Article  MathSciNet  MATH  Google Scholar 

  38. Macworth AK (1973) Interpreting pictures of polyhedral scenes. Artif Intell 4(2):121–137

    Article  Google Scholar 

  39. Metropolis N, Rosenbluth AW, Rosenbluth MN, Teller AH, Teller E (1953) Equation of state calculations by fast computing machines. J Chem Phys 21(6):1087–1092

    Article  Google Scholar 

  40. Ng AY, Jordan MI, Weiss Y (2001) On spectral clustering: Analysis and an algorithm. NIPS 14:849–856

    Google Scholar 

  41. Pearl J (1985) Heuristics. Intelligent search strategies for computer problem solving. The Addison-Wesley series in artificial intelligence, vol 1. Addison-Wesley, Reading. Reprinted version

    Google Scholar 

  42. Porway J, Zhu S-C (2011) Cˆ 4: exploring multiple solutions in graphical models by cluster sampling. IEEE Trans Pattern Anal Mach Intell 33(9):1713–1727

    Article  Google Scholar 

  43. Porway J, Wang Q, Zhu SC (2010) A hierarchical and contextual model for aerial image parsing. Int J Comput Vis 88(2):254–283

    Article  MathSciNet  Google Scholar 

  44. Potts RB (1952) Some generalized order-disorder transformations. In: Proceedings of the Cambridge philosophical society, vol 48, pp 106–109

    Article  MathSciNet  MATH  Google Scholar 

  45. Propp JG, Wilson DB (1996) Exact sampling with coupled Markov chains and applications to statistical mechanics. Random Struct Algor 9(1–2):223–252

    Article  MathSciNet  MATH  Google Scholar 

  46. Rao S, Tron R, Vidal R, Ma Y (2010) Motion segmentation in the presence of outlying, incomplete, or corrupted trajectories. IEEE Trans PAMI 32(10):1832–1845

    Article  Google Scholar 

  47. Rosenfeld A, Hummel RA, Zucker SW (1976) Scene labeling by relaxation operations. IEEE Trans Syst Man Cybern 6:420–433

    Article  MathSciNet  MATH  Google Scholar 

  48. Shi J, Malik J (2000) Normalized cuts and image segmentation. IEEE Trans Pattern Anal Mach Intell 22(8):888–905

    Article  Google Scholar 

  49. Sugihara K (1986) Machine interpretation of line drawings, vol 1. MIT Press, Cambridge

    Google Scholar 

  50. Sun D, Roth S, Black MJ (2010) Secrets of optical flow estimation and their principles. In: CVPR, pp 2432–2439

    Google Scholar 

  51. Swendsen RH, Wang J-S (1987) Nonuniversal critical dynamics in monte carlo simulations. Phys Rev Lett 58(2):86–88

    Article  Google Scholar 

  52. Tanner MA, Wong WH (1987) The calculation of posterior distributions by data augmentation. J Am Stat Assoc 82(398):528–540

    Article  MathSciNet  MATH  Google Scholar 

  53. Torralba A, Murphy KP, Freeman WT (2004) Sharing features: efficient boosting procedures for multiclass object detection. In: CVPR, vol 2. IEEE, pp II–762

    Google Scholar 

  54. Trefethen LN, Bau D III (1997) Numerical linear algebra, vol 50. SIAM, Philadelphia

    Book  MATH  Google Scholar 

  55. Tron R, Vidal R (2007) A benchmark for the comparison of 3-d motion segmentation algorithms. In: CVPR. IEEE, pp 1–8

    Google Scholar 

  56. Tu Z, Zhu S-C (2002) Image segmentation by data-driven Markov chain monte carlo. IEEE Trans Pattern Anal Mach Intell 24(5):657–673

    Article  Google Scholar 

  57. Vidal R, Hartley R (2004) Motion segmentation with missing data using power factorization and GPCA. In: CVPR, pp II–310

    Google Scholar 

  58. Von Luxburg U (2007) A tutorial on spectral clustering. Stat Comput 17(4):395–416

    Article  MathSciNet  Google Scholar 

  59. Weiss Y (2000) Correctness of local probability propagation in graphical models with loops. Neural Comput 12(1):1–41

    Article  MathSciNet  Google Scholar 

  60. Wolff U (1989) Collective monte carlo updating for spin systems. Phys Rev Lett 62(4):361

    Article  Google Scholar 

  61. Wu T, Zhu S-C (2011) A numerical study of the bottom-up and top-down inference processes in and-or graphs. Int J Comput Vis 93(2):226–252

    Article  MathSciNet  MATH  Google Scholar 

  62. Yan D, Huang L, Jordan MI (2009) Fast approximate spectral clustering. In: SIGKDD, pp 907–916

    Google Scholar 

  63. Yan J, Pollefeys M (2006) A general framework for motion segmentation: independent, articulated, rigid, non-rigid, degenerate and non-degenerate. In: ECCV, pp 94–106

    Google Scholar 

  64. Zhu S-C, Mumford D (2007) A stochastic grammar of images. Now Publishers Inc, Hanover

    MATH  Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Department of Statistics, Florida State University, Tallahassee, FL, USA

    Adrian Barbu

  2. Departments of Statistics and Computer Science, University of California, Los Angeles, Los Angeles, CA, USA

    Song-Chun Zhu

Authors
  1. Adrian Barbu
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Song-Chun Zhu
    View author publications

    You can also search for this author in PubMed Google Scholar

Rights and permissions

Reprints and permissions

Copyright information

© 2020 Springer Nature Singapore Pte Ltd.

About this chapter

Check for updates. Verify currency and authenticity via CrossMark

Cite this chapter

Barbu, A., Zhu, SC. (2020). Cluster Sampling Methods. In: Monte Carlo Methods. Springer, Singapore. https://doi.org/10.1007/978-981-13-2971-5_6

Download citation

Publish with us

Policies and ethics

Search

Navigation