Skip to main content
SpringerLink
Log in

Linear classifiers and selection of informative features

  • Mathematical Method in Pattern Recognition
  • Published:
Pattern Recognition and Image Analysis Aims and scope Submit manuscript

Abstract

In this work, to construct classifiers for two linearly inseparable sets, the problem of minimizing the margin of incorrect classification is formulated, approaches to achieving approximate solution, and calculation estimates of the optimal value for this problem, are considered. Results of computational experiments that compare proposed approaches with SVM are presented. The problem of identifying informative features for large-dimensional diagnostic applications is analyzed and algorithms for its solution are developed.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Similar content being viewed by others

References

  1. G. M. Fung and O. L. Mangasarian, “Multicategory proximal support vector machine classifiers,” Mach. Learn. 59, 77–97 (2005).

    Article  MATH  Google Scholar 

  2. Chih-Wei Hsu and Chih-Jen Lin, “A comparison of methods for multiclass support vector machines,” IEEE Trans. Neural Networks 13 (2), 415–425 (2002).

    Article  Google Scholar 

  3. J. C. Platt, N. Cristianini, and J. Shawe-Taylor, “Large margin DAG’s for multiclass classification,” in Advances in Neural Information Processing Systems (MIT Press, Cambridge, MA, 2000), Vol. 12, pp. 547–553.

    Google Scholar 

  4. Yu. Laptin, Yu. Zhuravlev, and A. Vinogradov, “A comparison of some approaches to classification problems, and possibilities to construct optimal solutions efficiently,” Pattern Recogn. Image Anal. 24 (2), 189–195 (2014).

    Article  Google Scholar 

  5. T. G. Dietterich and G. Bakiri, “Solving multiclass learning problems via error-correcting output codes,” J. Artif. Intellig. Res. 2, 263–286 (1995).

    MATH  Google Scholar 

  6. C. Lemaréchal, “Lagrangian relaxation,” Comput. Combinat. Optimiz. 2241, 112–156 (2001).

    MathSciNet  MATH  Google Scholar 

  7. L. Vanderberghe and S. Boyd, “Semidefinite programming,” Siam Rev., No. 38, 49–95 (1996).

    Article  MathSciNet  MATH  Google Scholar 

  8. N. Z. Shor, Nondifferentiable Optimization and Polynomial Problems (Kluwer Acad. Publ., Amsterdam, Dordrecht, London, 1998).

    Book  MATH  Google Scholar 

  9. S. Kim, M. Kojima, and H. Waki, “Exploiting sparsity in SDP relaxation for sensor network localization,” SIAM J. Optimiz. 20 (1), 192–215 (2009).

    Article  MathSciNet  MATH  Google Scholar 

  10. O. A. Berezovskyi, “Exactness criteria for SDP-relaxations of quadratic extremum problems,” Cybernet. Syst. Anal. 52 (6), 915–920 (2016).

    Article  MATH  Google Scholar 

  11. L. Vanderberghe and S. Boyd, “Semidefinite programming,” Siam Rev., No. 38, 49–95 (1996).

    Article  MathSciNet  MATH  Google Scholar 

  12. O. A. Berezovskyi, “On the accuracy of dual bounds for quadratic extremum problems,” Cybernet. Syst. Anal. 48 (1), 26–30 (2012).

    Article  MathSciNet  MATH  Google Scholar 

  13. V. V. Dmitrenko et al., “Determination of molecular glioblastoma subclasses on the basis of analysis of gene expression,” Cytol. Genet. 48 (6), 383–39 (2014).

    Article  Google Scholar 

  14. Yu. Laptin, “Exact penalty functions and convex extensions of functions in schemes of decomposition in variables,” Cybernet. Syst. Anal. 52, 85–95 (2016).

    Article  MathSciNet  MATH  Google Scholar 

  15. N. Z. Shor and N. G. Zhurbenko, “A minimization method using the operation of extension of the space in the direction of the difference of two successive gradients,” Cybernet. Syst. Anal. 7 (3), 450–459 (1971).

    Article  Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Dorodnicyn Computing Centre of the Russian Academy of Sciences, Moscow, 119333, Russia

    Yu. I. Zhuravlev & A. P. Vinogradov

  2. Glushkov Institute of Cybernetics of the Ukrainian National Academy of Sciences, Kiev, 03680, Ukraine

    Yu. P. Laptin, N. G. Zhurbenko, O. P. Lykhovyd & O. A. Berezovskyi

Authors
  1. Yu. I. Zhuravlev
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Yu. P. Laptin
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. A. P. Vinogradov
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. N. G. Zhurbenko
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. O. P. Lykhovyd
    View author publications

    You can also search for this author in PubMed Google Scholar

  6. O. A. Berezovskyi
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to A. P. Vinogradov.

Additional information

The article is published in the original.

Yurii Ivanovich Zhuravlev. Born in Voronezh, 1935. In 1957, graduated from Moscow State University. Doctor of Physics and Mathematics since 1965, professor since 1967, academician of Russian Academy of Sciences since 1992. Deputy director of Dorodnicyn Computing Centre, Federal Research Centre ‘Computer Science and Control’ of Russian Academy of Sciences, chair of the Mathematics Department of Russian Academy of Sciences, head of the Chair of Moscow State University, editor in chief of International journal Pattern Recognition and Image Analysis. Foreign member of Spanish Royal Academy of Sciences, National Academy of Sciences of Ukraine, and European Academy of Sciences. Winner of Lenin and Lomonosov prizes. Main fields of scientific interest: mathematical logic; control systems theory; mathematical theory of pattern recognition, data analysis and forecasting; operations research; artificial intelligence. Developed new research areas: theory of local optimization algorithms, algorithms of estimate calculation, algebraic theory of recognition algorithms.

Yurii Petrovich Laptin. Born in 1951. Graduated from the Moscow Institute of Physics and Technology in 1974. Doctor of Physics and Mathematics since 2016. Senior Researcher at the Department of Methods of Nonsmooth Optimization at the Glushkov Institute of Cybernetics, National Academy of Sciences of Ukraine. Scientific interests: methods of mathematical programming and their applications. Author of 70 scientific papers.

Aleksandr Petrovich Vinogradov. Born in 1951. Graduated from the Moscow Institute of Physics and Technology in 1974. Candidate of Physics and Mathematics since 1978. Senior Researcher at Dorodnicyn Computing Centre, Federal Research Centre ‘Computer Science and Control’ of Russian Academy of Sciences. Scientific interests: algebraic and geometric methods in pattern recognition and image processing. Author of 70 scientific papers.

Oleg Anatol’evich Berezovskyi. Born in 1964. Graduated from the Moscow Institute of Physics and Technology in 1988. Candidate degree in mathematical cybernetics, 1995. Senior Researcher at the Department of Nonsmooth Optimization Methods, Glushkov Institute of Cybernetics, National Academy of Sciences of Ukraine. Scientific interests: optimization methods and their applications. Author of 50 scientific papers.

Nikolai Georgievich Zhurbenko. Born in 1946. Graduated from the Moscow Institute of Physics and Technology in 1970. Candidate of Physics and Mathematics since 1976. Senior Researcher at the Department of Methods of Nonsmooth Optimization at the Glushkov Institute of Cybernetics, National Academy of Sciences of Ukraine. Scientific interests: methods of mathematical programming and their applications. Author of 80 scientific papers.

Oleksii Petrovich Lykhovyd. Born in 1959. Graduated from Kyiv Polytechnical Institute in 1983. Presently, Scientific Researcher at the Department of Nonsmooth Optimization Methods, Glushkov Institute of Cybernetics, National Academy of Sciences of Ukraine. Scientific interests: optimization methods and their applications. Author of 48 scientific papers.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Zhuravlev, Y.I., Laptin, Y.P., Vinogradov, A.P. et al. Linear classifiers and selection of informative features. Pattern Recognit. Image Anal. 27, 426–432 (2017). https://doi.org/10.1134/S1054661817030336

Download citation

  • Received:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1134/S1054661817030336

Keywords

Search

Navigation