Skip to main content
SpringerLink
Log in

Pattern Recognition Theory in Nonlinear Signal Processing

  • Published:
Journal of Mathematical Imaging and Vision Aims and scope Submit manuscript

Abstract

A body of research has developed within the context of nonlinear signal and image processing that deals with the automatic, statistical design of digital window-based filters. Based on pairs of ideal and observed signals, a filter is designed in an effort to minimize the error between the ideal and filtered signals. The goodness of an optimal filter depends on the relation between the ideal and observed signals, but the goodness of a designed filter also depends on the amount of sample data from which it is designed. In order to lessen the design cost, a filter is often chosen from a given class of filters, thereby constraining the optimization and increasing the error of the optimal filter. To a great extent, the problem of filter design concerns striking the correct balance between the degree of constraint and the design cost. From a different perspective and in a different context, the problem of constraint versus sample size has been a major focus of study within the theory of pattern recognition. This paper discusses the design problem for nonlinear signal processing, shows how the issue naturally transitions into pattern recognition, and then provides a review of salient related pattern-recognition theory. In particular, it discusses classification rules, constrained classification, the Vapnik-Chervonenkis theory, and implications of that theory for morphological classifiers and neural networks. The paper closes by discussing some design approaches developed for nonlinear signal processing, and how the nature of these naturally lead to a decomposition of the error of a designed filter into a sum of the following components: the Bayes error of the unconstrained optimal filter, the cost of constraint, the cost of reducing complexity by compressing the original signal distribution, the design cost, and the contribution of prior knowledge to a decrease in the error. The main purpose of the paper is to present fundamental principles of pattern recognition theory within the framework of active research in nonlinear signal processing.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Similar content being viewed by others

References

  1. J. Barrera, E.R. Dougherty, and N.S. Tomita, “Automatic Programming of Binary Morphological Machines by Design of Statistically Optimal Operators in the Context of Computational Learning Theory,” Electronic Imaging, Vol. 6, No. 1, pp. 54-67, 1997.

    Google Scholar 

  2. E.R. Dougherty and J. Barrera, “Logical Image Operators,” in Nonlinear Filters for Image Processing, eds. E. R. Dougherty and J. T. Astola, SPIE and IEEE Presses, Belingham, 1999.

    Google Scholar 

  3. E.R. Dougherty, S. Kim, and Y. Chen, “Coefficient of Determination in Nonlinear Signal Processing,” Signal Processing, Vol. 80, No. 10, pp. 2219-2235, 2000.

    Google Scholar 

  4. L. Devroye, L. Gyorfi, and G. Lugosi, A Probabilistic Theory of Pattern Recognition, Springer-Verlag, New York, 1996.

    Google Scholar 

  5. J. Barrera, E.R. Dougherty, and M. Brun, “Hybrid Human-Machine Binary Morphological Operator Design: An Independent Constraint Approach,” Signal Processing, Vol. 80, No. 8, pp. 1469-1487, 2000.

    Google Scholar 

  6. O.V. Sarca, E.R. Dougherty, and J. T. Astola, “Secondarily Constrained Boolean Filters,” Signal Processing, Vol. 71, No. 3, pp. 247-263, 1998.

    Google Scholar 

  7. O. Sarca, E.R. Dougherty, and J.T. Astola, “Two Stage Binary Filters,” Electronic Imaging, Vol. 8, No. 3, pp. 219-232, 1999.

    Google Scholar 

  8. N.T. Hirata, E.R. Dougherty, and J. Barrera, “Iterative Design of Morphological Binary Image Operators,” Optical Engineering, Vol. 39, No. 12, pp. 3106-3123, 2000.

    Google Scholar 

  9. E.R. Dougherty, “Optimal Mean-Square N-Observation Digital Morphological Filters — Part I: Optimal Binary Filters,” CVGIP: Image Understanding, Vol. 55, No. 1, pp. 36-54, 1992.

    Google Scholar 

  10. R.P. Loce and E.R. Dougherty, “Optimal Morphological Restoration: The Morphological Filter Mean-Absolute-Error Theorem,” Visual Communication and Image Representation, Vol. 3, No. 4, pp. 412-432, 1992.

    Google Scholar 

  11. E.R. Dougherty and R.P. Loce, “Optimal Mean-Absolute-Error Hit-or-Miss Filters: Morphological Representation and Estimation of the Binary Conditional Expectation,” Optical Engineering, Vol. 32, No. 4, pp. 815-823, 1993.

    Google Scholar 

  12. E. R. Dougherty and R. P. Loce, “Precision of Morphological Representation Estimators for Translation-Invariant Binary Filters: Increasing and Nonincreasing,” Signal Processing, Vol. 40, No. 3, pp. 129-154, 1994.

    Google Scholar 

  13. R.P. Loce and E.R. Dougherty, “Facilitation of Optimal Binary Morphological Filter Design via Structuring Element Libraries and Design Constraints,” Optical Engineering, Vol. 31, No. 5, pp. 1008-1025, 1992.

    Google Scholar 

  14. E.R. Dougherty, Y. Zhang, and Y. Chen, “Optimal Iterative Increasing Binary Morphological Filters,” Optical Engineering, Vol. 35, No. 12, pp. 3495-3507, 1996.

    Google Scholar 

  15. R. Hirata, E.R. Dougherty, and J. Barrera, “Aperture Filters,” Signal Processing, Vol. 80, No. 4, pp. 697-721, April, 2000.

    Google Scholar 

  16. E.R. Dougherty, “Optimal Mean-Square N-Observation Digital Morphological Filters — Part II: Optimal Gray-Scale Filters,” CVGIP: Image Understanding, Vol. 55, No. 1, pp. 55-72, 1992.

    Google Scholar 

  17. R.P. Loce and E.R. Dougherty, “Mean-Absolute-Error Representation and Optimization of Computational-Morphological Filters,” CVGIP: Image Understanding, Vol. 57, No. 1, 1995.

  18. E.J. Coyle and J.-H. Lin, “Stack Filters and the Mean Absolute Error Criterion,” IEEE Trans. on Acoustics, Speech, and Signal Processing, Vol. 36, No. 8, pp. 1244-1254, 1988.

    Google Scholar 

  19. J.T. Astola and P. Kuosmanen, “Representation and Optimization of Stack Filters,” Nonlinear Filters for Image Processing, eds. E. R. Dougherty and J. T. Astola, SPIE and IEEE Presses, Belingham, 1999.

    Google Scholar 

  20. M. Gabbouj and E.J. Coyle, “Minimum Mean Absolute Error Stack Filtering with Structuring Constraints and Goals,” IEEE Trans. Acoustics, Speech, and Signal Processing, Vol. 38, No. 6, pp. 955-968, 1990.

    Google Scholar 

  21. P. Kuosmanen and J.T. Astola, “Optimal Stack Filters Under Rank Selection and Structural Constraints, Signal Processing,” Vol. 41, No. 3, pp. 309-338, 1995.

    Google Scholar 

  22. E.R. Dougherty and D. Sinha, “Computational Mathematical Morphology,” Signal Processing, Vol. 38, pp. 21-29, 1994.

    Google Scholar 

  23. E.R. Dougherty and J. Barrera, “Computational Gray-Scale Operators,” Nonlinear Filters for Image Processing, eds. E. R. Dougherty and J. T. Astola, SPIE and IEEE Presses, Belingham, 1999.

    Google Scholar 

  24. E.R. Dougherty and D. Sinha, “Computational Gray-Scale Mathematical Morphology on Lattices (A Computer-Based Image Algebra). Part I: Architecture,” Real-Time Imaging, Vol. 1, pp. 69-85, 1995.

    Google Scholar 

  25. G.J.F. Banon and J. Barrera, “Minimal Representation for Translation-Invariant Set Mappings by Mathematical Morphology,” SIAM J. Applied Mathematics, Vol. 51, 1991.

  26. G.J.F. Banon and J. Barrera, “Decomposition of Mappings Between Complete Lattices by Mathematical Morphology, Part I. General Lattices,” Signal Processing, Vol. 30, 1993.

  27. L. Devroye, “Necessary and Sufficient Conditions for the Almost Everywhere Convergence of Nearest Neighbor Regression Function Estimates,” Zeitschrift fur Wahrscheinlichkeitstheorie und verwandte Gebiete, Vol. 61, pp. 467-481, 1982.

    Google Scholar 

  28. L. Gordon and R. Olshen, “Asymptotically Efficient Solutions to the Classification Problem, Annals of Statistics, Vol. 6, pp. 525-533.

  29. C. Stone “Consistent Nonparametric Regression,” Annals of Statistics, Vol. 5, pp. 595-645, 1977.

    Google Scholar 

  30. V. Vapnik and A. Chervonenkis, Theory of Pattern Recognition, Nauka, Moscow, 1974.

    Google Scholar 

  31. V. Vapnik and A. Chervonenkis, “On the Uniform Convergence of Relative Frequencies of Events to Their Probabilities,” Theory of Probability and its Applications, Vol. 16, pp. 264-280, 1971.

    Google Scholar 

  32. K. Alexander, “Probability Inequalities for Empirical Processes and a Law of the Iterated Logarithm,” Annals of Probability, Vol. 4, pp. 104-1067.

  33. L. Devroye and G. Lugosi, “Lower Bounds in Pattern Recognition and Learning,” Pattern Recognition, Vol. 28, pp. 1011-1018, 1995.

    Google Scholar 

  34. T. Cover, “Geometrical and Statistical Properties of Systems of Linear Inequalities with Applications in Pattern Recognition,” IEEE Trans. On Electronic Computers, Vol. 14, pp. 326-334, 1965.

    Google Scholar 

  35. K. Hornik, “Approximation Capabilities of Multi-layer Feedforward Networks,” Neural Networks, Vol. 4, pp. 251-257, 1991.

    Google Scholar 

  36. G. Cybenko, “Approximation by Superposition of Sigmoidal Functions,” Math. Control, Signals, Systems, Vol. 2, pp. 303-314, 1989.

    Google Scholar 

  37. K. Funahashi, “On the Approximate Realization of Continuous Mappings by Neural Networks,” Neural Networks, Vol. 2, pp. 183-192, 1989. Dougherty and Barrera 196

    Google Scholar 

  38. K. Hornik, M. Stinchombe, and H. White, “Multilayer Feedforward Networks are Universal Approximators,” Neural Networks, Vol. 2, pp. 359-366, 1989.

    Google Scholar 

  39. T. Chen, H. Chen, and R. Liu, “A Constructive Proof and an Extension of Cybenko's Approximation Theorem,” Proc. of the 22nd Symposium of the Interface: Computing Science and Statistics, 163-168, American Statistical Association, Alexandria, 1990.

    Google Scholar 

  40. E. Baum, “On the Capabilities of Multilayer Perceptrons,” Complexity, Vol. 4, pp. 193-215, 1988.

    Google Scholar 

  41. P. Bartlett, “Lower Bounds on the Vapnik-Chervonenkis Dimension of Multi-layer Threshold Networks,” Proc. Of the Sixth Annual ACM Conference on Computational Learning Theory, 144-150, Association for Computing Machinery, New York, 1993.

    Google Scholar 

  42. A. Farago and G. Lugosi, “Strong Universal Consistency of Neural Network Classifiers,” IEEE Trans. on Information Theory, Vol. 39, pp. 1146-1151, 1993.

    Google Scholar 

  43. J. Barrera, E.R. Dougherty, and N. S. Hirata, “Design of Optimal Morphological Operators Using Prior Filters,” Acta Stereologica, Vol. 16, No. 3, pp. 193-200, 1997.

    Google Scholar 

  44. E.R. Dougherty and R.P. Loce, “Optimal Binary Differencing Filters: Design, Logic Complexity, Precision Analysis, and Application to Digital Document Processing,” Electronic Imaging, Vol. 5, No. 1, pp. 66-86, January, 1996.

    Google Scholar 

  45. R.P. Loce and E.R. Dougherty, Enhancement and Restoration of Digital Documents: Statistical Design of Nonlinear Algorithms, SPIE Press, Bellingham, 1997.

    Google Scholar 

  46. E.R. Dougherty and J. Barrera, “Bayesian Design of Optimal Morphological Operators Based on Prior Distributions for Conditional Probabilities,” Acta Stereologica, Vol. 16, No. 3, pp. 167-174, 1997.

    Google Scholar 

  47. V.G. Kamat, E.R. Dougherty, and J. Barrera, “Multiresolution Bayesian Design of Binary Filters,” Electronic Imaging, Vol. 9, No. 3, pp. 283-295, 2000.

    Google Scholar 

  48. E.R. Dougherty, J. Barrera, G. Mozelle, S. Kim, and M. Brun, “Multiresolution Analysis for Optimal Binary Filters,” Mathematical Imaging and Vision, Vol. 14, No. 1, pp. 53-72, 2001.

    Google Scholar 

  49. A. V. Mathew, E.R. Dougherty and V. Swarnakar, “Efficient Derivation of the Optimal Mean-Square Binary Morphological Filter from the Conditional Expectation via a Switching Algorithm for the Discrete Power-Set Lattice,” Circuits, Systems, and Signal Processing, Vol. 12, No. 3, pp. 409-430, 1993.

    Google Scholar 

  50. N. Hirata, E.R. Dougherty, and J. Barrera, “A Switching Algorithm for Design of Optimal Increasing Binary Filters over Large Windows,” Pattern Recognition, Vol. 33, No. 6, pp. 1059-1081, 2000.

    Google Scholar 

  51. I. Tabus, D. Petrescu, and M. Gabbouj, “ATraining Framework for Stack and Boolean Filtering — Fast Optimal Design Procedures and Robustness Case Study,” IEEE Trans. on Image Processing, Vol. 5, No. 6, pp. 809-826, 1996.

    Google Scholar 

  52. I. Tabus and B. Dumitrescu, “A New Fast Method for Training Stack Filters,” IEEE-EURASIP Workshop on Nonlinear Signal and Image Processing, Antalya, Turkey, June 1999.

    Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Department of Electrical Engineering, Texas A&M University, USA

    Edward R. Dougherty

  2. Departamento de Ciencia de Computacao, Uinversidade de Sao Paulo, Brazil

    Junior Barrera

Authors
  1. Edward R. Dougherty
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Junior Barrera
    View author publications

    You can also search for this author in PubMed Google Scholar

Rights and permissions

Reprints and permissions

About this article

Cite this article

Dougherty, E.R., Barrera, J. Pattern Recognition Theory in Nonlinear Signal Processing. Journal of Mathematical Imaging and Vision 16, 181–197 (2002). https://doi.org/10.1023/A:1020325626071

Download citation

  • Issue Date:

  • DOI: https://doi.org/10.1023/A:1020325626071

Search

Navigation