Skip to main content
SpringerLink
Log in

Regularized fuzzy clusterwise ridge regression

  • Regular Article
  • Published:
Advances in Data Analysis and Classification Aims and scope Submit manuscript

Abstract

Fuzzy clusterwise regression has been a useful method for investigating cluster-level heterogeneity of observations based on linear regression. This method integrates fuzzy clustering and ordinary least-squares regression, thereby enabling to estimate regression coefficients for each cluster and fuzzy cluster memberships of observations simultaneously. In practice, however, fuzzy clusterwise regression may suffer from multicollinearity as it builds on ordinary least-squares regression. To deal with this problem in fuzzy clusterwise regression, a new method, called regularized fuzzy clusterwise ridge regression, is proposed that combines ridge regression with regularized fuzzy clustering in a unified framework. In the proposed method, ridge regression is adopted to estimate clusterwise regression coefficients while handling potential multicollinearity among predictor variables. In addition, regularized fuzzy clustering based on maximizing entropy is utilized to systematically determine an optimal degree of fuzziness in memberships. A simulation study is conducted to evaluate parameter recovery of the proposed method as compared to the extant non-regularized counterpart. The usefulness of the proposed method is illustrated by an application concerning the relationship among the characteristics of used cars.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Similar content being viewed by others

References

  • Aurifeille J-M, Quester PG (2003) Predicting business ethical tolerance in international markets: a concomitant clusterwise regression analysis. Int Bus Rev 12: 253–272

    Article  Google Scholar 

  • Belsley DA, Kuh E, Welsch RE (1980) Regression diagnostics: identifying influential data and sources of collinearity. Wiley, New York

    MATH  Google Scholar 

  • Bezdek JC (1981) Pattern recognition with fuzzy objective function algorithms. Plenum Press, New York

    MATH  Google Scholar 

  • Brusco MJ, Cradit JD, Steinley D, Fox GL (2008) Cautionary remarks on the use of clusterwise regression. Multivar Behav Res 43: 29–49

    Article  Google Scholar 

  • DeSarbo WS, Cron WL (1988) A maximum likelihood methodology for clusterwise linear regression. J Classif 5: 249–282

    Article  MATH  MathSciNet  Google Scholar 

  • DeSarbo WS, Oliver RL, Rangaswamy A (1989) A simulated annealing methodology for clusterwise linear regression. Psychometrika 54: 707–736

    Article  Google Scholar 

  • DeVeaux RD (1989) Mixtures of linear regressions. Comput Stat Data Anal 8: 227–245

    Article  MathSciNet  Google Scholar 

  • D’Urso P (2003) Linear regression analysis for fuzzy/crisp input and fuzzy/crisp output data. Comput Stat Data Anal 42: 47–72

    Article  MATH  MathSciNet  Google Scholar 

  • D’Urso P, Santoro A (2006) Fuzzy clusterwise linear regression analysis with symmetrical fuzzy output variable. Comput Stat Data Anal 51: 287–313

    Article  MATH  MathSciNet  Google Scholar 

  • Efron B (1982) The jackknife, the bootstrap, and other resampling plans. Society of Industrial and Applied Mathematics CBMS-NSF Monographs 38

  • Falk TH, Shatkay H, Chan W-Y (2006) Breast cancer prognosis via Gaussian mixture regression. Canadian conference on electrical and computer engineering, Queen’s University, Canada, pp 987–990

  • Gordon AD (1999) Classification. Chapman and Hall/CRC, Boca Raton

    MATH  Google Scholar 

  • Hathaway RJ, Bezdek JC (1993) Switching regression models and fuzzy clustering. IEEE Trans Fuzzy Syst 1: 195–204

    Article  Google Scholar 

  • Heiser WJ, Groenen PJF (1997) Cluster differences scaling with a within-cluster loss component and a fuzzy successive approximation strategy to avoid local minima. Psychometrika 62: 63–83

    Article  MATH  MathSciNet  Google Scholar 

  • Hoerl AE, Kennard RW (1970) Ridge regression: biased estimation for nonorthogonal problems. Technometrics 12: 55–67

    Article  MATH  Google Scholar 

  • Hoerl AE, Kennard RW (1970) Ridge regression: applications to nonorthogonal problems. Technometrics 12: 69–82

    Article  MATH  Google Scholar 

  • Hosmer DW (1974) Maximum likelihood estimates of the parameters of a mixture of two regression lines. Commun Stat 3: 995–1006

    Article  Google Scholar 

  • Hruschka H (1986) Market definition and segmentation using fuzzy clustering methods. Int J Res Mark 3: 117–134

    Article  Google Scholar 

  • Kuiper S (2008) Introduction to multiple regression: how much is your car worth? J Stat Educ 16. Available via http://www.amstat.org/publications/jse/v16n3/datasets.kuiper.html. Accessed 25 May 2009

  • Kutner MH, Nachtsheim C, Neter J (2004) Applied linear regression models. McGraw-Hill/Irwin, Boston

    Google Scholar 

  • Laviolette M, Seaman JW (1992) Evaluating fuzzy representations of uncertainty. Math Sci 17: 26–41

    MATH  MathSciNet  Google Scholar 

  • Laviolette M, Seaman JW (1994) Unity and diversity of fuzziness—from a probability viewpoint. IEEE Trans Fuzzy Syst 2: 38–42

    Article  Google Scholar 

  • Laviolette M, Seaman JW, Barrett JD, Woodall WH (1995) A probabilistic and statistical view of fuzzy methods. Technometrics 37: 249–261

    Article  MATH  Google Scholar 

  • Li R-P, Mukaidono M (1995) A maximum entropy approach to fuzzy clustering. Proceedings of the 4th IEEE international conference on fuzzy systems, Yokohama, Japan, pp 2227–2232

  • McBratney AB, Moore AW (1985) Application of fuzzy sets to climatic classification. Agric For Meteorol 35: 165–185

    Article  Google Scholar 

  • Miyamoto S (1998) An overview and new methods in fuzzy clustering. In: Jain LC, Jain RK (eds) Second international conference on knowledge-based intelligent electronic systems, Adelaide, Australia, pp 21–23

  • Miyamoto S, Mukaidono M (1997) Fuzzy c-means as a regularization and maximum entropy approach. 7th international fuzzy systems association world congress, Prague, Czech Republic, pp 86–92

  • Miyamoto S, Ichihashi H, Honda K (2008) Algorithms for fuzzy clustering: methods in c-means clustering with applications. Springer, Berlin

    MATH  Google Scholar 

  • Quandt RE (1958) The estimation of the parameters of a linear regression system obeying two separate regimes. J Am Stat Assoc 53: 873–880

    Article  MATH  MathSciNet  Google Scholar 

  • Quandt RE (1972) A new approach to estimating switching regressions. J Am Stat Assoc 67: 306–310

    Article  MATH  Google Scholar 

  • Quandt RE, Ramsey JB (1978) Estimating mixtures of normal distributions and switching regressions. J Am Stat Assoc 73: 730–738

    Article  MATH  MathSciNet  Google Scholar 

  • Späth H (1979) Algorithm 39: clusterwise linear regression. Computing 22: 367–373

    Article  MATH  MathSciNet  Google Scholar 

  • Späth H (1981) Correction to algorithm 39: clusterwise linear regression. Computing 26: 275

    Article  MATH  MathSciNet  Google Scholar 

  • Späth H (1982) Algorithm 48: a fast algorithm for clusterwise linear regression. Computing 29: 175–181

    Article  MATH  Google Scholar 

  • Späth H (1985) Cluster dissection and analysis. Wiley, New York

    MATH  Google Scholar 

  • Takane Y, Hwang H (2007) Regularized linear and kernel redundancy analysis. Comput Stat Data Anal 52: 394–405

    Article  MATH  MathSciNet  Google Scholar 

  • Tran D, Wagner M (2000) Fuzzy entropy clustering. The 9th IEEE international conference on fuzzy systems, San Antonio, pp 152–157

  • Wedel M, DeSarbo WS (1995) A mixture likelihood approach for generalized linear models. J Classif 12: 21–55

    Article  MATH  Google Scholar 

  • Wedel M, Kistemaker C (1989) Consumer benefit segmentation using clusterwise linear regression. Int J Res Mark 6: 45–49

    Article  Google Scholar 

  • Wedel M, Steenkamp J-BEM (1989) A fuzzy clusterwise regression approach to benefit segmentation. Int J Res Mark 6: 241–258

    Article  Google Scholar 

  • Wedel M, Steenkamp J-BEM (1991) A clusterwise regression method for simultaneous fuzzy market structuring and benefit segmentation. J Mark Res 28: 385–396

    Article  Google Scholar 

  • Yang M-S, Ko C-H (1997) On cluster-wise fuzzy regression analysis. IEEE Trans Syst Man Cybern B 27: 1–13

    Article  Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Department of Psychology, McGill University, 1205 Dr. Penfield, Montreal, QC, H3A 1B1, Canada

    Hye Won Suk & Heungsun Hwang

Authors
  1. Hye Won Suk
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Heungsun Hwang
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Hye Won Suk.

Rights and permissions

Reprints and permissions

About this article

Cite this article

Suk, H.W., Hwang, H. Regularized fuzzy clusterwise ridge regression. Adv Data Anal Classif 4, 35–51 (2010). https://doi.org/10.1007/s11634-009-0056-5

Download citation

  • Received:

  • Revised:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11634-009-0056-5

Keywords

Mathematics Subject Classification (2000)

Search

Navigation