Symmetry and antisymmetry properties of optimal solutions to regression problems

Abstract

Besides requiring a good fit of the learned model to the empirical data, machine learning problems usually require such a model to satisfy additional constraints. Their satisfaction can be either imposed a-priori, or checked a-posteriori, once the optimal solution to the learning problem has been determined. In this framework, it is proved in the paper that the optimal solutions to several batch and online regression problems (specifically, the Ordinary Least Squares, Tikhonov regularization, and Kalman filtering problems) satisfy, under certain conditions, either symmetry or antisymmetry constraints, where the symmetry/antisymmetry is defined with respect to a suitable transformation of the data. Computational issues related to the obtained theoretical results (i.e., reduction of the dimensions of the matrices involved in the computations of the optimal solutions) are also described. The results, which are validated numerically, have potential application in machine-learning problems such as pairwise binary classification, learning of preference relations, and learning the weights associated with the directed arcs of a graph under symmetry/antisymmetry constraints.