Elsevier

Neurocomputing

Volume 232, 5 April 2017, Pages 3-15
Neurocomputing

Clustering techniques for Fuzzy Cognitive Map design for time series modeling

https://doi.org/10.1016/j.neucom.2016.08.119Get rights and content

Abstract

This study presents an approach to time series modeling with Fuzzy Cognitive Maps. In the paper we focus on initial modeling phase: map nodes selection. The research objective was to introduce algorithmic means to evaluate Fuzzy Cognitive Map design before training phase. We posed a hypothesis that application of cluster validity indexes could serve us in this endeavor. In order to validate the proposed approach we have conducted a suite of experiments on various time series, both synthetic and real-world. Five cluster validity indexes turned out to be especially valuable in our study. Results show that Fuzzy Cognitive Maps designed using one of the five selected indexes have superior quality. First, they are easy to interpret, because map nodes are related with the underlying data points. Second, after we train such maps, it turns out that the numerical quality of their predictions outrivals maps with other designs.

Introduction

Understanding and modeling complex phenomena in our environment is among the most important problems that we face. There is a need not only for numerical approaches that express knowledge in a form of raw numbers, but also for qualitatively-oriented methods that provide description of information in a human-centered way, coherent with the way how we perceive and learn.

Though giant strides have been made in the recent decades in brain studies, yet there are many unraveled mysteries. Humans do not think in a binary, “raw”, machine-alike way. Instead, we acquire and process knowledge in a form of concepts. Acknowledging it leads to formulation of modeling frameworks that aim at mimicking a concepts-based knowledge representation system. Cognitive Maps, and in particular a family of Cognitive Maps named Fuzzy Cognitive Maps (FCMs), are examples of such approaches.

FCM represents knowledge with a signed directed graph. Nodes in the map correspond to abstract concepts, edges to the relationships between them. In an FCM-based model we also evaluate the levels of concepts activation. By iterative exploration of the map, we observe how concepts' states change in response to various input factors and interactions between them. In other words, we learn structures in data. Such information representation scheme provides an inherent ability to model and forecast phenomena.

The research reported in this paper is built upon the premise that FCMs provide necessary means to represent complex phenomena in an intuitive way, similar to the way in which human beings think and reason. Let us stress here the role of linguistic labels, which we attach to FCM nodes in order to enhance human-centeredness of the model.

We take under investigation the time series modeling procedure, in which map nodes correspond to phenomenon levels. Examples of linguistic labels for such nodes are: “low level of phenomenon”, “moderate”, “high”, etc. We postulate that the number of nodes in the map (i.e. map design) should be learned from data. Having in mind that the number of nodes is the elementary factor determining ease of interpretation we present an experimental study, where we employ cluster validity indexes to evaluate different map designs.

The primary objective of our research is to provide a comprehensive, algorithmically-aided methodology for time series modeling using FCMs. Map design is a crucial step of the modeling procedure that despite its importance is often neglected. The particular aim of this study is to introduce a new FCM design technique based on cluster validity indexes that allows constructing models that represent time series well. This method equips the modeler with a measure for FCM evaluation that can be employed before the training process. So far the modeling methodology that we work with required repeating the training procedure in order to compare several ready models and select the best one. To avoid such inconvenience, we propose a novel data-driven FCM evaluation scheme. We show that it enhances the modeling process, helps constructing good maps and allows avoiding repetitions of the computationally costly training procedure. The contribution of our study is very practical: we present a suite of cluster validity indexes that perform well in the task of FCM nodes evaluation.

We admit that employing cluster validity indexes in a domain that at the first sight seems to be unfamiliar may raise questions. Let us argue that in fact there are elementary similarities between FCM design and clustering. The FCM-based time series modeling method that we develop expresses causal relationships between concepts that correspond to different levels of a phenomenon. Let us give a very simple example: say we have an FCM with three nodes corresponding to low, moderate, and high level of precipitation. The data that we used to construct the map has been gathered on a daily basis. The trained map could give us a prediction in the following form: if today we have, say, low precipitation then tomorrow we will most likely have, say, moderate precipitation. Connections in the FCM (edges in the map) that let us predict daily precipitation have been learned from actual time series data. The original time series, which typically is a very long sequence of numerical values, has been embedded in a particular coordinates system. Properties of the coordinates system, i.e. the modeling space and its dimensionality, play crucial role. Given example was on purpose a trivial case, recalled for illustrative purposes. Detecting regularities in the modeling space leads to extraction of meaningful concepts that generalize numerous data points. In consequence, we may replace inconvenient numerical values with general concepts. It becomes apparent that we do not know what would be the optimal number of concepts to represent a given dataset. The more concepts we have, the more detailed predictions we can make. At the same time, too many concepts cause confusion when we interpret the model. In this perspective, we propose to apply cluster validity indexes to evaluate various FCM designs. In a series of empirical experiments we show that the proposed approach lets us to form maps with superior quality.

The paper is organized as follows. In Section 2 we introduce basic notions related to FCMs. Section 3 presents the methodology of interest for time series modeling. Section 4 addresses the issues of map design and presents our approach. Section 5 presents experiments. Finally, Section 6 covers conclusions.

Section snippets

Knowledge modeling with Fuzzy Cognitive Maps – preliminaries

FCMs appeared in the literature in 1986 in [1]. B. Kosko, the father of FCMs, inspired by the works of a political scientist R. Axelrod [2] proposed a novel soft approach to knowledge modeling that soon gained a substantial popularity.

FCMs are an important class of models expressing hazy sequential linkages among concepts. An example of an FCM with three nodes is in Fig. 1.

An FCM based on c concepts is represented with a weighted directed graph. Concepts, also called nodes, A1,A2,,Ac are

Time series modeling with Fuzzy Cognitive Maps

In our research we refer to the approach to time series analysis with FCMs outlined in [3]. A time series is recognized as a scalar sequence:z1,z2,z3,,zMwhere zi is a real number and M is the number of data points in the scalar time series. The time series is transformed into a two-dimensional space by computing increment values: (zi,dzi), where dzi=zizi1. Now, the time series is a sequence of pairs, where the start index is two, because the change of amplitude cannot be set together with

Design considerations

Extraction of concepts for an FCM is a pivotal design phase. Proper concepts and time series representation induce a good model. The goal of modeling with FCMs is to extract generalized relationships in the data. Hence, the objective is to select abstract concepts that represent the information in aggregated fashion and cover the dataset. It is worth to emphasize that once we have found a proper set of concepts we may start describing the entire dataset (that could be a so-called big data time

Environment

In the study we have applied various cluster validity indexes for FCM design evaluation. First, we elaborate in detail on two case examples. Subsequently, we present less detailed results for further datasets.

Experiments have been executed in R environment. The following external R libraries have been used: pso [26] for executing the Particle Swarm Optimization procedure, e1071 [27] for the fuzzy c-means algorithm, clusterCrit [25] for cluster validity indexes, dbscan [28] for the DBSCAN

Conclusion

The article discusses a comprehensive framework for time series modeling with FCMs. We have built upon an existing approach to time series modeling with FCMs and introduced a method for map design. In particular, we have shown that optimal FCM design, i.e. a collection of concepts that make the map, entails a superior model. The benefits of a well-designed map are twofold. First, it is the easiness of its interpretation and second, a satisfying numerical quality of predictions. We have employed

Acknowledgments

The research is supported by the National Science Center, grant No 2012/07/B/ST6/01501, decision No DEC-2012/07/B/ST6/01501.

Wladyslaw Homenda received the M.Sc. and Ph.D. degrees from Warsaw University of Technology, Warsaw, Poland, and the D.Sc. degree from the System Research Institute of Polish Academy of Sciences, Poland. He is currently an Associate Professor with the Faculty of Mathematics and Information Science, Warsaw University of Technology. He is also with the Faculty of Economics and Informatics in Vilnius (Lithuania) of the University of Bialystok, Poland. His main research interests are in theoretical

References (31)

  • V. Georgopoulosa et al.

    A fuzzy cognitive map approach to differential diagnosis of specific language impairment

    Artif. Intell. Med.

    (2003)
  • W. Stach, L. Kurgan, W. Pedrycz, A survey of fuzzy cognitive map learningmethods, Issues in Soft Computing, Theory and...
  • E. Papageorgiou et al.

    Fuzzy cognitive maps learning using particle swarm optimization

    J. Intell. Inf. Syst.

    (2005)
  • K. Parsopoulos, E. Papageorgiou, P. Groumpos, M. Vrahatis, A first study of fuzzy cognitive maps learning using...
  • J. Kennedy, R. Eberhart, Particle swarm optimization, in: Proceedings of the IEEE International Conference on Neural...
  • Cited by (0)

    Wladyslaw Homenda received the M.Sc. and Ph.D. degrees from Warsaw University of Technology, Warsaw, Poland, and the D.Sc. degree from the System Research Institute of Polish Academy of Sciences, Poland. He is currently an Associate Professor with the Faculty of Mathematics and Information Science, Warsaw University of Technology. He is also with the Faculty of Economics and Informatics in Vilnius (Lithuania) of the University of Bialystok, Poland. His main research interests are in theoretical foundations of computer science, knowledge representation and processing and intelligent computing technologies, specifically in the areas of man-machine communication and human-centric computing, fuzzy modeling and Granular Computing, knowledge discovery and data mining. He currently serves as an Associate Editor of Information Sciences and is a member of several editorial boards of other international journals. More information can be found at http://www.mini.pw.edu.pl/~homenda/

    Agnieszka Jastrzebska received her M.Sc. Eng. degree in Computer Engineering from the Rzeszow University of Technology, Poland and Ph.D. degree from the Warsaw University of Technology, Poland. She is research and teaching assistant at the Faculty of Mathematics and Information Science at the Warsaw University of Technology. In the past 4 years she was awarded prestigious scholarships from the Institute of Computer Science of Polish Academy of Sciences, the Systems Research Institute of Polish Academy of Sciences, and the Center for Advanced Studies of Warsaw University of Technology. Her main research interests include machine learning, computational intelligence, and fuzzy modeling.

    View full text