Learning noise

In this paper we propose a genetic programming approach to learning stochastic models with unsymmetrical noise distributions. Most learning algorithms try to learn from noisy data by modeling the maximum likelihood output or least squared error, assuming that noise effects average out. While this process works well for data with symmetrical noise distributions (such as Gaussian observation noise), many real-life sources of noise are not symmetrically distributed, thus this approach does not hold. We suggest improved learning can be obtained by including noise sources explicitly in the model as a stochastic element. A stochastic element is a random sub-process or latent variable of a hidden system that can propagate nonlinear noise to the observable outputs. Stochastic elements can skew and distort output features making regression of analytical models particularly difficult and error minimizing approaches inhibiting. We introduce a new method to infer the analytical model of a system by decomposing non-uniform noise observed at the outputs into uniform stochastic elements appearing symbolically inside the system. Results demonstrate the ability to regress exact analytical models where stochastic elements are embedded inside nonlinear and polynomial hidden systems.