A Vision Architecture Based on Fiber Bundles

• ¹ Frankfurt Institute for Advanced Studies, Germany

We offer a conceptual framework for the construction of an artificial visual system. It serves as basis for the integration of sub-modalities (texture, depth, motion, color, contours, segment markers, illumination, ..) and for autonomous learning. The architecture will decisively reduce the effort necessary to achieve better visual functionality.

Visual sub-modalities are conveniently formulated as two-dimensional sheets of neurons with local connectivity. Such sheets can be formalized as manifolds. The points of a sheet carry internal feature spaces of different dimensionality (texture, e.g., may be formalized as a space of Gabor responses, stereo depth, motion and color as 1D, 2D and 3D spaces, respectively), constituting what mathematicians call fiber bundles. Manifolds admit spatial coordinate frames. Different sets of manifolds carry retinal, object-centered or scene-centered coordinate frames.

Manifolds are connected by mappings (which in the brain are realized by massive sets of axons). These mappings connect not only coordinate points but also feature spaces. For pairs of manifolds with different coordinate frames mappings are dynamical, that is, they change to keep track of relative motion. Dynamic mappings are governed by local control spaces, which encode relative position and motion. Dynamic mappings thus are themselves fiber bundles.

Mappings between fiber bundles implement constraints between their feature values. These constraints connect feature values that refer to the same point in the environment. When looking at a point on the surface of a moving body, for instance, a kinematic constraint connects values for depth and motion of the point, translational and rotatory motion of the body acting as latent parameters. The representation of local surface point motion, in turn, serves to dynamically control the mapping between manifolds with retinal and in object-centered coordinates.

Visual input leaves many variables uncertain due to imprecise, ambiguous or absent information. The sub-modality spaces reflect that in terms of more or less broad probability distributions. There is a large number of constraints between different qualities referring to the same point in an image. Many of these constraints are implemented in the mappings between fiber bundles. These mappings typically are forming loops, such that a given variable value is influenced (strengthened or weakened) by several converging mappings. Under the influence of these constraint signals, probability distributions dynamically collapse to delta functions. Thus, the system has contractor dynamics on the short-term level of image interpretation.

The system has contractor dynamics also on the long-term level of learning, converging on detailed circuits in a coarse-to-fine manner. Topological mappings between neural sheets have long been shown to be subject to attractor dynamics. Mismatches in constraint loops are repaired by adapting them to each other. Models of recurring patterns are stored by plasticity in horizontal connections.