DISTRIBUTED, LAYERED AND RELIABLE COMPUTING NETS TO REPRESENT NEURONAL RECEPTIVE FIELDS

. Receptive fields of retinal and other sensory neurons show a large variety of spatiotemporal linear and non linear types of responses to local stimuli. In visual neurons, these responses present either asymmetric sensi-tive zones or center-surround organization. In most cases, the nature of the responses suggests the existence of a kind of distributed computation prior to the integration by the final cell which is evidently supported by the anatomy. We describe a new kind of discrete and continuous filters to model the kind of computations taking place in the receptive fields of retinal cells. To show their performance in the analysis of different non-trivial neuron-like structures, we use a computer tool specifically programmed by the authors to that effect. This tool is also extended to study the effect of lesions on the whole performance of our model nets.

1. Introduction.The retina, as well as cortex, shows a conspicuous layered struc ture which strongly suggests they perform a kind of distributed layered computa tion [8], [9].In this structure there is always overlapping of "sensory fields" between the neighbouring units, so a double process of convergence-divergence of the infor mation takes place.Visual neurons both in retina and cortex present in addition either asymmetric sensitive zones or center-surround organization of their receptive fields [28], [13].In any case, the nature of the responses suggests the existence of a kind of distributed computation prior to the integration by the final cell, a fact evidently supported by the anatomy [36].
As the anatomy clearly shows, these computation intermediate "modules" are neither homogenous nor exactly regularly distributed, though, for the sake of for mulations, they are many times admitted to be so and referred to as "computing modules" or "subunits" [29], [23].
A general distributed discrete computing structure that generates the spatial weighting profiles of receptive fields has been previously described [19].We present here a general structure of layered distributed computation formed by two or three inputs and one output, by which any parallel layered computing structure can be represented.Functional formulations are made by Newton Filters, in the discrete case and generalized to Hermite Functions in the continuous.It is shown that these provide for appropriate representations of center-periphery receptive fields, both symmetric and asymmetric and that, in addition, any arbitrary weight profile can be represented by them.Also, some types of non linear, mostly local, computations are included.
By means of the computer tool developed to study large non-trivial neuron-like structures (for one dimensional or two dimensional radially symmetric arbitrary receptive fields profiles), it has been possible to perform both the analysis (given a structure, find the overall weighting function) and the synthesis (find a structure that performs a given function).Also, what is most relevant to distributed computation, the effect of "holes" or "local lesions" (scotomas) in different parts of the net are considered.For the cases studied, the effects of lesions are found to depend on their location and affect mostly the quantitative (amplitude) aspects of the weights than the qualitative (shape) ones.The results are reported here, as well as other consequences of the distributed nature of the computation.
This paper is organized in three important sections: first, discrete Newton Filters are described to model layered and distributed computation that takes place at retinal cells level.Second, Newton Filters extension to the continuum leads to Hermite functions and functionals.Third, the analysis and synthesis of weight profiles is described using a computer tool specifically programmed to that effect.

2.
Receptive fields: Basic layered, distributed structures.Prototypes of idealized structures corresponding to the retina or to cortex, show two important characteristics from the computational point of view.First, there are granular structures arranged in layers of quasi similar units, which project their normally many inputs to the single output from units of prior layer.That is, from one layer to the next there is a convergence of the information [11].Second, the single output of one unit in one layer is "picked up" by various or many units of the next.That is, there is also a divergence of the information.If the overall number of inputs lines to the net and the number of output lines (or their information content) are preserved (like in foveal zones), then all the incident local information could be recovered from the outputs, in despite that the local details have been "dispersed" by divergence and "mixed up" by convergence.Also, local lesions do not provoke "blind spots" or scotomas; only a kind of reduced resolution.The types of structures that we consider have these properties.
We start from the structure in figure 1 where a row of input lines correspond to a kind of linear receptive (sensory) field, providing for 20 signals, x i .There are three layers of computing units that will finally end in the output units O j .We must insist that there is not a one-to-one correspondence between computing units and neurons in modelling.Computing units could be "distributed" in, for example, the dendrites of real neurons, and/or in lateral connections, like those provided by amacrines and horizontal cells.Also, there is not such a one-to-one correspondence with formal or artificial neurons in ANN.
Convergent and divergent paths for signals can be easily identified in this network.For example, x 1 to x 13 signals converge into O 1 , whereas all the overlapping, from There is a simple multilayer representation for this net in which all the computing units of one layer are computing the same functions.This is shown in figure 2. Each unit has two inputs and connect to only two units of the next layer.Again, convergent and divergent paths can be easily identified.The interesting point is that both structures are equivalent, they have the same function and also that, given one net, the other can be obtained and viceversa.Notice that the number of layers is the same that the extent of the receptive field minus one (19 in the example).
We shall develop the analytical representations of the distributed nets for neuronal receptive fields by means of the 2 and 3 input lines units, and show for them the corresponding analysis and synthesis constructive theorems.Notice that the network of figure 1 could be reduce to 2 and 3 input units partially (per layers) or globally (represented by the network in figure 2).
Two/three inputs units representation networks lead naturally to Newton Filters for the discrete case and to Hermite Functions for the continuous cases.They permit clear and useful expressions for the weight profiles (filter kernels) and rapid solutions for the direct and inverse network problems.They also provide for a mean to investigate the effect of lesions in layered-parallel structures and a transparent way to introduce typical local non-linearities in visual processing networks.
3. Representation by Newton filters.Newton Filters can be introduced [21], [22] in the following way: Let us suppose a set of n inputs in one dimension: x 1 , x 2 , ... , x n and two rules, A and D, to combine two by two those inputs.We shall assume hereafter that A is the adding rule and D is the substracting rule, so that the output produced by inputs x i and x i+1 is as follows: and D(XJ + I, Xj) = Xj + i -Xj A(x i+1 ,x i ) = x i+1 + x i Other definitions for A and D are discussed in [21].In this way, n -1 outputs will be produced from n inputs, those n -1 outputs will produce n -2 outputs in turn and so on.Then we can study the way every input, x i , contributes to the final output, O.
For example, let us suppose 4 inputs x 1 , x 2 , x 3 , x 4 and the A rule (see figure 3).It can be observed that the weights, or contributions of each input x i (i = 1,2, 3, 4) to the final output O, are respectively 1, 3, 3, 1.
[  It is not difficult to see that, in general, the weight with which contributes the element at position k in row n is precisely the coefficient of the Newton Binomial expansion.In the previous example, the input x 2 which is in position k = 2, contributes in row n = 3 with a weight If, in the example above, instead of using the rule A, we use the rule D or subtraction, weights in absolute value remain the same, with signs alternating (see figure 4).
In this case, the weights are respectively 1,-3, 3, -1.In general, the input in position k in row n contributes to the output with the coefficient of the Newton Binomial expansion, that is,

( -1) n+ ( n \ k
This structure has a discrete filter nature.So, let us define a Newton Filter as the result of a cascade of processes A and D and let us denoted it as: where m and n stands for the number of rows with A and D processes respectively.The order in which these A and D processes are applied doesn't change the weights in the output.For a proof, see [21].For example, the filter N(A 2 , D 2 ), i.e., the Newton filter with two adding and two subtracting rows, is easily obtained as shown in figure 5.
Each square box in figure 5(a) can be considered as a computing subunit with two inputs-one output.Changing the squares to nodes, the equivalent representation of figure 5(b) is obtained.The weights are assumed to be +1 if it is not otherwise indicated.Using this procedure the overall weights for any filter can be computed.For example, the weights corresponding to the filter N(A 10 ,D 2 ) are: 1x 1 +8x 2 +26x 3 +40x 4 +15x 5 -48x 6 -84x 7 -48x 8 +15x 9 +40x 10 +26x 11 +8x 12 +1x 13

fx
The similarity between this computational structure and the retinal receptive fields weights for some ganglia is apparent, but for one dimension.In this model, the filter structure gives a kind of structure of the dendritic arborization of retinal cells.The 'tree' will be more complex as wider the filter is and, in this manner, the computing complexity will increase.
Classical center-periphery receptive field structures [1], [10], [25] can be obtained by the difference of discrete Gaussians, result of Newton Filters of different receptive field length.However, as it will be discussed, a better representation is obtained by Newton Filters where two layers are of local weights +1,-1, as illustrated in filter of figure 6.
It should be noted that, in the above model, all the micro-processes have weights with absolute value equal to 1 and it is from the contribution of each one of them where we get an overall weight distribution that is similar to that found in natural systems.
We next consider the general case of weights of the type (a, 6), with a, 6 real numbers, to obtain what we call "Generalized Newton Filters".
That is: where By putting local weights in the form cij • (1, ej) and if cij = 1, a normalized filter is obtained.
How can the global weights be computed from the local ones?Suppose we have 4 inputs xo,xi,X2,X3 in a normalized filter characterized by the weights (1, ei); (1,62); (1,63) for the rows 1, 2 and 3 respectively.For the first row we would have For the second row, Finally, for the third row, It is clear that the global weights are the sum of the products of the combinations of local weights taken 0 by 0, 1 by 1, 2 by 2, etc.In general, if we have a discrete filter with weights (wo, wi,..., w n ) and construct a normalization u>o' wo ' wo we obtain the micro-processes weights of the Generalized Newton Filter from the solutions of the following system of equations: In general, the solutions of this non-linear system of equations do not have to be real numbers for arbitrary Wj.It follows that by the relationship between the coefficients and the roots of a polynomial of n degree, the solutions, ej, of this system are the roots of the polynomial: When solving the inverse problem, that is, going from a set of given receptive field weights, Wj, to local units weights, ej, complex roots can not be accepted as possible weights in a Generalized Newton Filter.In this case, pairs of complex roots of the form (a+ j/3) and (a -j/3) correspond to a three input computing unit having real weights [(a 2 + /3 2 ), 2a, 1], as illustrated in figure 7.

=
(oc+P Figure 7. Reduction of complex roots for real representations.
As a conclusion, it is shown that any linear layered computation net having arbitrary receptive field weights is equivalent or can be represented by a multilayered net composed of two and three inputs units, which corresponds to a representation of a Generalized Newton Filter.And again, the multilayered net of 2-3 inputs units can be broken down to any smaller number of layers, where each computing unit of the resulting layers compute local Generalized Newton Filter weights, on receptive fields that overlap.

3.1.
Representation by Hermite functions.It has been proven elsewhere that the extension of the Newton Filters introduced in previous sections to the continuum leads to Hermite functions and functionals [21].Limit Theorems [4] show that weights for the Newton Filter N(A n ), i.e., ,, ( n\ , iv (A n ) = , 0 < x < n x appropriately normalized, converge, for n large, to the gaussian factor: Similarly, weights in the Newton Filter N(A n , D\) converge to the first derivative of /(x), i.e.,

f'(x) = -2x • e~x
Filters N(A n , Dj) with j = 2,..., m at the limit when n is large enough, produce the successive derivatives of /(x): These functions are the Hermite functions, H, multiplied by a ( -1)™ factor.The polynomials accompanying the gaussian factor e~x are the so called Hermite Polynomials [31], [7], for which a generating function is (Rodrigues' formula): ) n e x -e~x dx n As we can observe in figures 8(a), 8(b), 8(c), 8(d) and 8(e) there is a high similarity between those profiles and the sensitivities describing the neuronal receptive fields, mostly retinal ganglion cells [28], [13], [34], [35].We have both ON-OFF (negativepositive) center-periphery structures and structures alternating ON and OFF zones, when time is included.
The above can be extended to two dimensions by the bidimensional kernels, where H n (x) = z-e-t* +y ] H m (y) = e~^x +y ] ox n oy m For rotationally symmetric (radial) fields, the weighting Hermite profile of order n is given by: dr n where r is the distance to the center and the order n indicates the number of inhibitory layers in the microstructure [21], [19].The second order Hermite profile is precisely the "mexican hat", that since David Marr's Laplacian of Gaussians [16], has been accepted as a better center-periphery "visual filter" rather than the alternative difference of Gaussians [26], [30].
For R 0 the radius of the excitatory center, the radial second order Hermite profile is: W('*) = (1 -r /i? 0 ) • expy -r /2i? 0 ] with r 2 = x 2 + y 2 .This is shown in figure 9 Real receptive fields of, say, retinal ganglion cells, are at the end, the result of the convergence of different layers of other retinal cells.Though these receptive fields are better represented by structures like layers of discrete Newton filters shown before, the continuum representations given by Hermite structures provide for good insight to functions in space-time, that could afterwards be retranslated to discrete structures.
That is the case when there are delays or lasting effects due, for example, to relatively long lasting depolarizing and hyperpolarizing effects in center-periphery receptive fields, where directionality appears [2].
In the case of the Mexican hat profile of figure 9(a) for a local stimulus crossing diametrally the receptive field of the cell, and for relatively long lasting depolarizinghyperpolarizing actions, the cumulative activity of the cell, when the stimulus reaches point r, is proportional to the integral, from the left border (-co) to r, of W2(r), that is, which is the Hermite profile of order 1.A c (r) is shown in figure 9(b).It can be seen there that the cumulative activity is excitatory only during the centrifugal motion of the stimulus (right lobule of figure), even if the motion is started at the center, because the cumulative activity is null there.Notice that the above structure can correspond to a whole ganglion cell.In this case the cell will show a centrifugal directional selectivity, in addition to other local ON-OFF, contrast detector properties.Time ago, classical centripetally directional selective retinal ganglion cells (besides being locally ON-OFF) have been described for Group 2 ganglion cells in frogs [17], [18].It would correspond to an inhibitorycenter, excitatory-surround situation, that is, to -W(r).For global receptive fields of ganglion cells there is a possibility of a "multiple ring" structure alternating excitatory-inhibitory zones [6], [12], [32].This is representable by radial Hermite functions of higher order.Figure 10(a) shows a fourth Hermite weighting kernel, corresponding to the even radial kernel Wi(r) = (r -6r + 3) • exp[ -r /2] which consists of a center excitatory zone surrounded by an inhibitory ring and an additional far excitatory ring area.Again, when lasting excitation and inhibition are included, there appear peculiar directional sensitivity effects.The cumulative activity A c for a diametrally crossing stimulus is which is the third order Hermite profile, shown in figure 10(b).Notice that for said stimulus, there is centripetal selectivity at the first moment, to became centrifugal selectivity as the stimulus leaves the receptive field.
When the kernel or weighting function is not of rotational symmetry there are a variety of potential directionality properties, depending on the latencies of the excitatory and inhibitory areas.The preferred direction points from the excitatory to the inhibitory zones.The simplest case which corresponds to a single inhibitory layer in the x direction for the Newton Filters representation, is the Hermite bidimensional kernel of order 1, given by ) ox This kernel is represented in figure 11, where a preferred direction and an optimum position and orientation for a bright edge detection are shown.
Zonal directional selectivity appears when there are inhibitory layers in the x and/or y directions of the microstructure represented by the bidimensional discrete Newton Filters.In the continuum generalization, to each inhibitory microstructure in one direction corresponds a partial derivative, giving rise to the corresponding two dimensional Hermite kernel.
For example, for one inhibitory layer in each direction x and y of the microstructure, the resulting kernel is: This kernel is represented in figure 12. Favored directions are from excitatory to inhibitory zones.Notice that there is a null direction normal to an optimum direction, as it is indicated in the figure.Also, it shows a "corner" type optimum detection (bright corners edges entering the optimum direction).
As an additional illustration of the potentials of two dimensional Newton Filters and the corresponding continuum Hermite kernels, let us consider the case of two inhibitory layers in the x direction and two in the y direction.It corresponds to the bidimensional Hermite kernel of order four: This kernel is shown in figure 13, where one can see the existence of a centre excitatory area surrounded by a ring with alternating excitatory and inhibitory zones, showing a more complex pattern of optimum positions and orientations for edges and preferred directions.Though Hermite functions are a very appropriate analytical representation for sensorial neuronal receptive fields, in many cases this is only valid for small ranges, and also excluding non-linearities.
However, their microstructural substrate, provided by discrete Newton Filters, point to convenient generalizations to the continuum to cover situations like the -4 -4 Figure 13.Fourth order Hermite kernel with a ring of alternating excitatory and inhibitory zones.
"widening" of the inhibitory periphery in center-periphery quasi-linear neurons [33]; the non linear interactions between excitatory and inhibitory subfields and the existence of sustained linear and non linear responses.
These refinements, do not change the nature of the asymmetries due to non rotational Hermite kernels but provide for better approaches to the experimental results.
The main objection to Marr's proposal of the second derivative of a Gaussian as a spatial model for linear ganglia, comes from two facts: first, the surrounding OFF or inhibitory ring is much wider in real cells than it is predicted by said filter [33].Widening of the inhibitory may be a consequence of lowering the resolution from center to periphery in real receptive fields and its Newton Filter representations.In the case of the Hermite kernel representation this would be a kind of widening the scale for the r coordinate as going from center to periphery.Figure 14 illustrates this effect for an expansion of the type r' = r 1 ' 3 .
The second effect is the "no null" response to uniform illumination.This other non linearities are due to the interaction of excitatory and inhibitory components of signals, which can give rise to sustained responses under constant uniform stimulation.This is the case for a local realistic rectifying non linearity of the form:  [20] can provide both for the analysis and the synthesis of receptive fields in one dimension.That is, given local weights A of the units, one weight per layer, the global weighting of the terminal computing unit is found and vice versa, from a given (arbitrary, real numbers) string (row) of weights, that is, a kind of receptive field "mask", the application finds the weighting associated to each layer.

Simulations. A computer tool developed
In the process of calculating the global weighting profile, the application allows to introduce the number of inputs x i and the local weights per layer in the form of pairs (a i , b i ) or (1, e i ).Once computed, the application shows the global weights in a grid and the graphic of the weighting profile in two windows: one with a horizontal linear scale and the other with a non linear horizontal scale, that can be used to simulate the lowering of resolution from center to periphery.
The inverse problem, that is, given a weighting profile, to find the local weights, is also solved by the tool.As it was shown in section 3, this process implies finding the roots of a polynomial of degree n (see equation 2), by the Jenkins-Traub algorithm [24].The inverse problem is illustrated in figure 16(b) for an arbitrary receptive field sensitivity (amplitudes corresponding to the profile of a landscape in Gran Canaria).
A sample of the corresponding local weights is in figure 16(c) where it can be seen that most of them correspond to three-input units (the third value, which is 1, is omitted in the figure).This tool has been extended to include 'holes' or lesions (scotomas) in arbitrary parts of the net [3].As it would be expected from the topology of the net, the lesions affect the computational profile differently as they are produced closer to the final node (or cell body) or in the (few) inhibitory layers.The topology of the net could be changed to other than triangular form, as it happens in real neurons.
Figures 17(a) and 17(b) illustrate the effect of lesions in a dendritic-like structure of 80 input lines, with four inhibitory layers.They show the changes in the weighting function (kernel) after the lesions indicated in the left part of the figures.The black lines in the triangular structure show the position of inhibitory layers.The heavy line in graphics corresponds to the weighting function after lesion.
Figures 18(a) and 18(b) illustrate the effect of scotomas also close to the cell body.In this case, the local lateral inhibition takes place in the outer computational layers, in a number of 15 layers in each case.Comparing with results in figure 17, it can be seen that the nature of the computation is less affected when lateral inhibition  takes place in outer layers (early in the net).Also, the sensitivity to higher spatial frequencies is less affected, the main loose being of amplitude sensitivity.
5. Conclusions and discussion.Newton Filters and Hermite functions have been originally introduced to model processes and computations that take place in the receptive field of retinal cells.Given these computational structures, the analysis and synthesis of receptive fields is straightforward: given the microstructure of a receptive field the weight pattern can be found and, given a receptive field with a given weight profile, one can have the microstructure having that weight profile as its output.These results have also been extended to study the reliability of the computing nets, that is, the performance of the net in the presence of holes or lesions in such nets.These results should be connected with the reorganization of the receptive fields of neurons found after lesions [5,27].The linearity of the discrete filters presented here appears as a limitation of our formulation.However, the most relevant computing non linearities that have been documented in the physiological literature are local nonlinearities of the "feed forward" type, where classical methods of analysis (as Liapunov second method) have no sense of application.Rather, Wiener-Volterra or white noise and wavelet type of analysis are more appropriate although their formulations and theory do not provide any insight into the fine structural (microstructure) of the net.Since Marmarelis [14,15], white noise and other input-output systems theoretical types of analysis seem to show that the relevant is a type of diode effect.In general, our proposal allows for the introduction of nonlinearities of the global general type that have been experimentally found, for the discrete and continuous settings, and this should be further explored.In general, nonlinearities distributed at the intermediate layers of the net cannot be resolved into a closed mathematical form for the output computing agent, and computer simulation will be required.But even then, Newton Filters representations are a good starting frame, as it was pointed in the text.Distributed nonlinearities can be sometimes approximated by summarizing functionals at the output.
It must be remarked that all references to real neurons are highly and strictly paradigmatic, since our proposals are at the level of systems theory.We are not considering coding herein.All in all, results analytically obtained in this paper contribute to the hard task of describing the complex processes that take place in living organisms, given their similarity to those records obtained by artificial stimulation of cells.

Figure 1 .
Figure 1.A general three layers computing network.

Figure 2 .
Figure 2. Two inputs-one output computing units representation of a multilayered network, equivalent to the one in figure 1.

Figure 8 .
Figure 8.The first five Hermite functions.

6 Figure 9 .
Figure 9. (a) Center-periphery structure of receptive field, corresponding to the second order Hermite profile.(b) Cumulative activity from the field in (a) for long lasting excitatory-inhibitory effects, for a local stimulus moving diametrally from left to right.Note that excitatory activity only appears during centrifugal motion.

Figure 10 .
Figure 10.(a) A double ring receptive field, represented by a 4th order Hermite profile.Note the central excitatory area, surrounded by a first inhibitory ring and a second excitatory one.(b) The cumulative activity for a local stimulus moving diametrally from left to right, provokes first a sensitivity to centripetal motion and then a centrifugal selectivity.

Figure 11 .
Figure 11.Preferred direction and optimum position (orientation) by a first order bidimensional Hermite kernel, corresponding to one microstructure inhibition in the x direction.

Figure 12 .
Figure 12. Second order Hermite kernel showing normal nulloptimum directions and preferred zones for optimum position and orientation.

2 f 6 Figure 14 .
Figure 14.Widening of the inhibitory ring (continuous line) as a consequence of lowering the resolution from center to periphery by a factor r' = r 1 ' 3 .

F 6 Figure 15 .
[H(x,y)] = [exp(kH) -1], k < 1 Figure 15 illustrates changes in the kernel for the Mexican hat H2 and k = 0.4.Notice the decrease in amplitude of the inhibitory ring, resulting in a non-zero positive value for the mean value of kernel F. Decreasing of the amplitude for the inhibitory ring (continuous line) in the case of the H 2 kernel.

Figure 16 .
Figure 16.Outputs from the tool.(a) Total weight profile for Newton Filter N(A 10 , D 4 ).(b) Arbitrarily chosen receptive field weights profile.(c) Sample of three-inputs units weights of the net.

Figure 16 (
Figure 16(a) shows a part of the main screen view of the tool for a case similar to that in figure 2 with 15 input lines and local weights from filter N(A 10 , D 4 ).

Figure 17 .
Figure 17.Effects of lesions in a dendritic-like structure of 80 input lines.Lesion in layer 78 with four subtractive layers at (a) the middle of the structure and (b) the end of the structure.

Figure 18 .
Figure 18.Illustration of the effect of scotomas in late layers (close to the cell body) when the computational peculiarities (in this case, lateral inhibition) takes place in more outer layers.Figure (a) corresponds to the first 15 layers being inhibitory and the scotomas (three) in layer 96, for a "retina" of 100 inputs.Figure (b) corresponds to the same situation with two scotomas.
Figure 18.Illustration of the effect of scotomas in late layers (close to the cell body) when the computational peculiarities (in this case, lateral inhibition) takes place in more outer layers.Figure (a) corresponds to the first 15 layers being inhibitory and the scotomas (three) in layer 96, for a "retina" of 100 inputs.Figure (b) corresponds to the same situation with two scotomas. X