Skip to main content
SpringerLink
Log in

Parallel Nonnegative Matrix Factorization Algorithm on the Distributed Memory Platform

  • Published:
International Journal of Parallel Programming Aims and scope Submit manuscript

Abstract

Nonnegative matrix factorization provides a new sight into the observed signals and has been extensively applied in face recognition, text mining and spectral data analysis. Despite the success, it is inefficient for the large-scale data set, due to the notoriously slow convergence of the multiplicative updating method. In this paper, we try to solve the problem through the parallel computing technique. Considering the limitation of the shared memory platform, the parallel algorithms are implemented on the distributed memory platform with the message passing interface library. Moreover, we adopt the two-layer cascade factorization strategy to eliminate the network consumption. The parallel implementations are evaluated on a 16-node Beowulf cluster with two data sets in different scale. The experiments demonstrate that the proposed method is effective in both precision and efficiency.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Similar content being viewed by others

References

  1. Lee D.D., Seung H.S.: Learning the parts of objects by non-negative matrix factorization. Nature 401, 788–791 (1999)

    Article  Google Scholar 

  2. Lee D.D., Seung H.S.: Algorithms for non-negative matrix factorizations. Adv. Neural Inf. Process. Syst. 13, 556–562 (2001)

    Google Scholar 

  3. Berry M.W., Browne M.: Email surveillance using non-negative matrix factorization. Comput. Math. Organ. Theory 11, 249–264 (2005)

    Article  MATH  Google Scholar 

  4. Berry M.W., Browne M., Langville A.N., Pauca V.P., Plemmons R.J.: Algorithms and applications for approximate non-negative matrix factorization. Comput. Stat. Data Anal. 52, 155–173 (2007)

    Article  MATH  MathSciNet  Google Scholar 

  5. Kaarna, A.: Nonnegative matrix factorization features from spectral signatures of AVIRIS images. In: Proceedings of IEEE International Conference on Geoscience and Remote Sensing Symposium, pp. 549–552 (2006)

  6. Pauca V.P., Piper J., Plemmons R.J.: Nonnegative matrix factorization for spectral data analysis. Linear Algebra Appl. 416(1), 29–47 (2006)

    Article  MATH  MathSciNet  Google Scholar 

  7. Wang, B., Zhou, H., Zhang, L.M.: Blind decomposition of mixed pixels using constrained non-negative matrix factorization. In: Proceedings of IEEE International Conference on Geoscience and Remote Sensing Symposium, vol. 6, pp. 3757–3760 (2005)

  8. Miao, L.D., Qi, H.R.: A constrained non-negative matrix factorization approach to unmix highly mixed hyperspectral data. In: Proceedings of IEEE International Conference on Image Processing, vol. 2, pp. 185–188 (2007)

  9. Hoyer P.O.: Nonnegative matrix factorization with sparseness constraints. J. Mach. Learn. Res. 5, 1457–1469 (2004)

    MathSciNet  Google Scholar 

  10. Li, S.Z., Hou, X.W., Zhang, H.J., Cheng, Q.S.: Learing spatially localized, parts-based representation. In: Proceedings of IEEE Computer Society Conference on Computer Vision and Pattrern Recognition, vol. 1, pp. 207–212, Hawaii, USA (2001)

  11. Hyvärinen A., Oja E.: Independent component analysis: algorithms and applications. Neural Networks 13, 411–430 (2000)

    Article  Google Scholar 

  12. Cichocki A., Zdunek R.: Multilayer nonnegative matrix factorization using projected gradient approcaches. Int. J. Neural Syst. 17, 431–446 (2007)

    Article  Google Scholar 

  13. Cichocki A., Zdunek R.: Multilayer nonnegative matrix factorization. Elect. Lett. 42, 947–948 (2006)

    Article  Google Scholar 

  14. Vasiloglou, N., Gray, A.G., Anderson, D.V.: Non-negative matrix factorization, convexity and isometry. In: Proceedings of the Ninth SIAM International Conference on Data Mining, pp. 673–684 (2009)

  15. Wild S., Curry J., Dougherty A.: Improving non-negative matrix factorizations through structured initialization. Pattern Recogn. 37, 2217–2232 (2004)

    Article  Google Scholar 

  16. Wild, S.: Seeding non-negative matrix factorizations with the spherical kmeans clustering. Master’s Thesis, Department of Applied Mathematics, University of Colorado (2003)

  17. Boutsidis C., Gallopoulos E.: SVD based initialization: a head start for non-negative matrix factorization. Pattern Recogn. 41, 1350–1362 (2008)

    Article  MATH  Google Scholar 

  18. Lin, C.-J.: Projected gradient methods for non-negative matrix factorization. Technical Report Information and Support Services (Technical Report ISSTECH-95-013), Department of Computer Science, National Taiwan University (2005)

  19. Gonzalez, E.F., Zhang, Y.: Accelerating the Lee-Seung algorithm for non-negative matrix factorization. Technical Report TR-05-02, Department of Computational and Applied Mathematics, Rice University (2005)

  20. Zdunek, R., Cichocki, A.: Nonnegative matrix factorization with quasi-newton optimization. In: Proceedings of the 8th International Conference on Artificial Intelligence and Soft Computing, ICAISC, Zakopane, Poland, June 25–29 (2006)

  21. Robila, S.A., Maciak, L.G.: A parallel unmixing algorithm for hyperspectral images. In: Proceedings of SPIE Conference on Intelligent Robots and Computer Vision XXIV: Algorithm, Techniques and Active Vision, vol. 6384, Boston, USA (2006)

  22. Robila, S.A., Maciak, L.G.: Sequential and parallel feature extraction in hyperspectral data using non-negative matrix factorization. In: Proceedings of IEEE Long Island Conference on Systems, Applications and Technology, pp. 1–7 (2007)

  23. The message passing interface standard: http://www-unix.mcs.anl.gov/mpi

  24. Thakur R., Rabenseifner R., Gropp W.: Optimization of collective communication operations in MPICH. Int. J. High Perform. Comput. Appl. 19, 49–66 (2005)

    Article  Google Scholar 

  25. NMFLAB for Signal and Image Processing: http://www.bsp.brain.riken.jp/ICALAB/nmflab.html

  26. Green, R.O., Chrien, T.G., Nielson, P.J., Sarture, C.M., Eng, B.T., Chovit, C., Murray, A.T., Eastwood, M.L., Novack, H.I.: Airborne visible/infrared imaging spectrometer (AVIRIS): recent improvements to the sensor and data facility. In: Proceedings of SPIE Conference on Imaging Spectrometry of the Terrestrial Environment, vol. 1937, pp. 180–190, Orlando, USA (1993)

  27. LINPACK: http://www.netlib.org/linpack

  28. Basic linear algebra subprograms: http://www.netlib.org/blas

  29. Winter, M.E.: N-FINDR an algorithm for fast autonomous spectral end-member determination in hyperspectral data. In: Proceedings of SPIE Conference on Imaging Spectrometry V, vol. 3753, pp. 266–275, Denver, USA (1999)

  30. Kruse, F.A., Boardman, J.W., Huntington, J.F., Mason, P., Quigley, M.A.: Evaluation and validation of EO-1 Hyperion for geologic mapping. In: Proceedings of IEEE International Conference on Geoscience and Remote Sensing Symposium, vol. 1, pp. 593–595 (2002)

  31. Nascimento J.M.P., Dias J.M.B.: Vertex component analysis: a fast algorithm to unmix hyperspectral data. IEEE Trans. Geosci. Remote Sens. 43(4), 898–910 (2005)

    Article  Google Scholar 

  32. Clark, R.N., Swayze, G.A., Gallagher, A.J., King, T.V.V., Calvin, W.M.: The U.S. geological survey, digital spectral library: version 1: 0.2–3.0 microns. U.S. Geological Survey Open File Report, pp. 93–592 (1993)

Download references

Author information

Authors and Affiliations

  1. School of Instrument Science and Opto-electronics Engineering, Beijing University of Aeronautics and Astronautics, XueYuan Road No. 37, HaiDian District, 100191, Beijing, China

    Chao Dong, Huijie Zhao & Wei Wang

Authors
  1. Chao Dong
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Huijie Zhao
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Wei Wang
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Chao Dong.

Rights and permissions

Reprints and permissions

About this article

Cite this article

Dong, C., Zhao, H. & Wang, W. Parallel Nonnegative Matrix Factorization Algorithm on the Distributed Memory Platform. Int J Parallel Prog 38, 117–137 (2010). https://doi.org/10.1007/s10766-009-0116-7

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s10766-009-0116-7

Keywords

Search

Navigation