Journal of Graph Algorithms and Applications Threshold Circuits Detecting Global Patterns in Two-dimensional Maps

In this paper, we consider a biologically-inspired Boolean function P n D that models a simple task of detecting global spatial patterns on a two-dimensional map. We prove that P n D is computable by a threshold circuit of size (i.e., number of gates) O(√ n log n), which is an improvement on the previous upper bound O(n), while our circuit has larger depth O(√ n) and total wire length O(n log 2 n). Moreover, we demonstrate that the size of our circuit is nearly optimal up to a logarithmic factor: we show that any threshold circuit computing P n D has size Ω(√ n/ log n).


Introduction
A threshold circuit is a combinatorial circuit comprising logic gates computing linear threshold function, and comprises one of the most well-studied computational models in circuit complexity theory.It is known that threshold circuits have surprising computational power: polynomial-size and constant-depth threshold circuits can compute a variety of Boolean functions including basic arithmetic operations such as addition, multiplication, division, sorting, etc. [6,8,9,11,10].Finding an explicit function that cannot be computed by polynomial-size and constant-depth threshold circuits is a standard approach to resolving the well-known P vs. NP problem, and threshold circuits are currently on its cutting edge (e.g., see [12]).
Threshold circuits have been studied from another perspective: threshold circuits are considered to provide a theoretical model of neural networks in the brain [6,7,10].It is known that a threshold gate, the basic element of a threshold circuit, captures the basic input-output characteristic of a biological neuron.In this line of research, we expect to take a step towards understanding how a neural network realizes complicated information processing (e.g., sensory processing) with remarkably high speed and low energy consumption.
However, classical circuit complexity theory provides little insight; Legenstein and Maass suggested the following two reasons [2].(1) Conventional complexity theory focuses on a different set of computational problems such as the arithmetic operations above mentioned, and (2) standard complexity measures such as circuit size and depth are not tailored to resources that are of primary interest in neuromorphic engineering and the analysis of neural circuits in biological organisms.Motivated by this viewpoint, Legenstein and Maass attempted to resolve the situation by introducing the following setting.
They proposed several biologically-inspired Boolean functions that model simple tasks of visual information processing.They consider the so-called local feature detectors in neural networks, in which a local feature detector is a biological unit detecting the presence of a salient local feature, such as a center that emits higher intensity than its surrounding, a line segment in a particular direction, or even more complex local visual patterns such as an eye or a nose.Intuitively, they define their Boolean functions as problems of asking whether two (or more) types of local feature detectors form a predefined simple configuration.In addition, being motivated by the importance of minimizing wiring in determining brain organization (e.g., see [1] and its references), they propose a complexity measure, total wire length, that is the sum of the lengths of wires in a particular circuit implementation model.They investigate threshold circuits computing their Boolean functions by means of the total wire length [2,3,4].
Among the proposed Boolean functions, we focus on one of them, called P n D , defined on a two-dimensional map on which local feature detectors of two types reflecting spatial relationship in the outside world are arranged.P n D is a simple pattern detection problem querying whether there exists a pair of different local feature detectors satisfying a simple condition: one local feature is detected below and to the left of the other (see Section 2.3 for precise definition).

Ref.
size depth total wire length Table 1: Complexities of prior and our circuits.We omit some advantages of the circuits provided in [2].The first circuit comprises AND/OR gates with just two inputs and outputs.The second circuit is formulated in terms of ∆, 2 ≤ ∆ ≤ √ n, which is the maximum number of inputs of the gates.The third circuit comprises threshold gates with at most O(log n) inputs.
Legenstein and Maass showed that P n D is computable by a circuit of reasonably small size, depth, and total wire length [2].They provide a circuit of size (i.e., number of gates) O(n), depth O(log n • log log n), and total wire length O(n), in which every gate computes the AND or OR of two inputs.They then consider the case in which AND/OR gates are permitted to have larger inputs and total wire length and show that P n D is computable by a circuit of size O(n), depth O(log n/ log ∆), and total wire length O((n • ∆ • log n)/ log ∆) for any integer ∆, 2 ≤ ∆ ≤ √ n, in which every gate computes AND or OR of at most ∆ inputs.They also give a threshold circuit of size (i.e., the number of gates) O(n), depth O(log n), and total wire length O(n), in which the threshold gates have O(log n) inputs.(See Table 1.) Our main result in this paper is an improvement on the size of threshold circuits computing P n D .We prove that P n D is computable by a threshold circuit of size O( √ n log n), depth O( √ n), and total wire length O(n log 2 n).We obtain our circuit by construction, and our proof exhibits the explicit structure of the desired circuit.We note that our circuit comprises a number of copies of subcircuits performing the same task with a different set of inputs.
An optimal size of threshold circuits computing P n D is of independent interest.In fact, as a complement to our circuit construction, we further show that any threshold circuit computing P n D is of size Ω( √ n/ log n).Thus, the size of our circuit is optimal up to a polylogarithmic factor.
The remainder of the paper is organized as follows.In Section 2, we define threshold circuits and the complexity measures, and provide a formal definition of P n D .In Section 3, we show that P n D is computable by a threshold circuit of O( √ n log n) gates.In Section 4, we provide a lower bound Ω( √ n/ log n) .In Section 5, we finally conclude with some remarks.

Definitions
In Section 2.1, we define threshold circuits.In Section 2.2, we define three complexity measures, size, depth, and total wire length.In Section 2.3, we provide the precise definition of P n D .

Threshold Circuits
A threshold gate with an arbitrary number z of inputs computes a linear threshold function with z inputs: for every input x = (x 1 , x 2 , . . ., x z ) ∈ {0, 1} z , the output g(x) of a threshold gate g with integer weights w 1 , w 2 , . . ., w z and threshold t is defined as where for any number η, sign(η) = 1 if η ≥ 0, sign(η) = 0, otherwise.
A threshold circuit C is a combinatorial circuit of threshold gates, and is defined by a directed acyclic graph.Let n be the number of input variables to C. Then each node of in-degree 0 in C corresponds to one of the n input variables x 1 , x 2 , . . ., x n , and the other nodes correspond to threshold gates.
Let C be a threshold circuit with n input variables x 1 , x 2 , . . ., x n and s gates.Let g 1 , g 2 , . . ., g s be the gates in C, which are topologically ordered with respect to the underlying directed acyclic graph of C. We regard the output of g s as the output C(x) of C, and call the gate g s the top gate of C.

Complexity Measures for Threshold Circuits
We use three complexity measures, size, depth, and total wire length for threshold circuits.The first two measures are standard in circuit complexity; for a threshold circuit C, the size s(C) of C, simply denoted by s, is the number of gates in C, and the depth d(C) of C, simply denoted by d, is the number of gates on the longest directed path to the top gate of C on its underlying directed acyclic graph.
Legenstein and Maass proposed the last measure, total wire length, to evaluate threshold circuits from the biological viewpoint.The measure is based on the following model of a circuit implementation that abstracts the total wire length of a neural network in cortical circuitry.(See [2] and its references for justification.More precisely, we adopt Model (A) in [2].) We assume that gates, input-ports and output-ports are placed on intersection points on a two-dimensional grid on which two adjacent points are unitdistance apart.We can connect these gates and ports by routing wires in the Euclidean plane; wires can cross and need not run rectilinearly.If a gate g has k inputs, then g occupies a set of k intersection points that are all connected by an undirected wire.Any of these k points can be used to provide one of the k Figure 1: Arrangement of indices of x = (x 0,0 , x 0,1 , . . ., x σ,σ ) ∈ {0, 1} n and y = (y 0,0 , y 0,1 , . . ., y σ,σ ) ∈ {0, 1} n on the 2-dimensional grid.inputs or one of the outputs of the function computed by g.A wire is permitted to branch and provide inputs to the other gates.We define total wire length of a circuit C as the minimal value of the sum of all the wire lengths of any such arrangement implementing C.

Construction of Circuit
In this section we give an upper bound on the size of threshold circuits computing P n D , as in the following theorem.We prove the theorem by construction.In Section 3.1, we define some terms, and obtain a useful lemma.In Section 3.2, we construct the circuit.In Section 3.3, we evaluate the size, depth and total wire length of the circuit.

Construction of C
Based on Lemma 3.1, we now construct the desired threshold circuit C. Let τ = log √ n for simplicity.First, for each j, 0 ≤ j ≤ σ, we construct a set of τ threshold gates g t j , 0 ≤ t ≤ τ − 1, such that the outputs of τ gates represent max(x; * , j) in binary system.We utilize a circuit construction used in [2].For each pair of i, 0 ≤ i ≤ σ, and t, 0 ≤ t ≤ τ , let Clearly, if the (t + 1)st bit of the binary representation of i is 1, then p(i, t) is odd, thus u(i, t) = 2 i ; and otherwise, p(i, t) is even, thus u(i, t) = −2 i .Let j, 0 ≤ j ≤ σ, be an arbitrarily fixed index.For each t, 0 ≤ t ≤ τ − 1, the gate g t j has threshold 1, and receives every input in the jth column: g t j receives x i,j with weight u i,t for every i, 0 ≤ i ≤ σ.Thus, for every x ∈ {0, 1} n , Equations ( 9) and ( 10) imply that max(x; * , j) and thus equals to the (t + 1)st bit of the binary representation of max(x; * , j).
Second, for each j, 0 ≤ j ≤ σ, we similarly construct a set of τ threshold gates h t j , 0 ≤ t ≤ σ, such that the outputs of the τ gates represent min(y; * , j) in the binary system.For each t, 0 ≤ t ≤ τ − 1, the gate h t j has threshold 1, and receives every input in the jth column: h t j receives y i,j with weight for every i, 0 ≤ i ≤ σ.Thus, for every y ∈ {0, 1} n , Then Eqs. ( 12) and (13) imply that min(y; * , j) and hence the output of h t i equals to the (t+1)st bit of the binary representation of min(y; * , j).
Using h t j , 0 ≤ t ≤ τ − 1, we construct gates ĥt j , 0 ≤ t ≤ τ − 1, that represent min j≤q≤σ min(y; * , q); Figure 3: Overview of the circuit for ĥt j , 0 ≤ t ≤ τ − 1, where the gates labeled with "∧" compute the AND of two inputs and the gates labeled with "∨" computes the OR of two inputs.the construction is inductive on j from σ to 0. For the case in which j = σ, we do not create any new gate, and simply identify ĥt σ with h t σ for every t, 0 ≤ t ≤ τ − 1, since min σ≤q≤σ min(y; * , q) = min(y; * , σ).
For each j, σ − 1 ≥ j ≥ 1, we introduce two gates a j , b j and 2τ gates a t j and b t j , 0 ≤ t ≤ τ −1, whose outputs are used for inputs to ĥt j , 0 ≤ t ≤ τ −1.The gates a j and b j determine whether min(y; * , j) is smaller than min j+1≤q≤σ min(y; * , q), and are defined as By Eq. ( 14), a j outputs 1 if and only if min(y; * , j) < min j+1≤q≤σ min(y; * , q); whereas b j outputs 1 if and only if min(y; * , j) ≥ min j+1≤q≤σ min(y; * , q); Thus, exactly one of a j and b j outputs 1 for any input.Then, for each t, 0 ≤ t ≤ τ − 1, the gate a t l computes the AND of the outputs of h t j and a j : Clearly, the output of a t j is equal to that of h t j if a j (y) = 1 (i.e.min(y; * , j) < min j+1≤q≤σ min(y; * , q)); and equals to 0, otherwise.Similarly, the gate b t j computes the AND of the outputs of h t j+1 and b j : The output of b t j is equal to that of ĥt j+1 if b j (y) = 1 (i.e.min(y; * , j) ≥ min j+1≤q≤σ min(y; * , q)); and equals to 0, otherwise.For each t, 0 ≤ t ≤ τ − 1, we obtain the gate ĥt j simply computing the OR of the outputs of a t j and b t j ; ĥt j (y) = sign(a t j (y) + b t j (y) − 1).
Clearly, we have See Fig. 3 which depicts the circuit for computing ĥt j , 0 ≤ t ≤ τ − 1.Finally, we construct σ + 1 gates r 0 , r 1 , . . ., r σ such that for each j, 0 ≤ j ≤ σ − 1, the gate r j determines whether there exists an index j satisfying Eq. (1).Equations ( 11) and (15) imply that we can obtain such r j , as follows: r j has threshold 0 and receives the outputs of g t j with weight 2 t and ĥt j+1 with weight −2 t for every t, 0 ≤ t ≤ τ − 1.More formally, Consequently, Lemma 3.1 implies that P n D (x, y) = 1 if and only if there exists an index j, 0 ≤ j ≤ σ −1, such that r j (x, y) = 1.Therefore, our construction of C is completed by adding the top gate s computing the OR of r 0 , r 1 , . . ., r σ−1 :

Analysis of C
We now analyze the size, depth and total wire length of C.
For every j, 0 ≤ j ≤ σ, we have 3τ gates g t j , h t j and ĥt j , 0 ≤ t ≤ τ − 1.In addition, for every j, 0 ≤ j ≤ σ, we have 2τ gates a t j and b t j , 0 ≤ t ≤ τ − 1 together with two gates a j and b j .Besides, we have r 0 , r 1 , . . ., r σ−1 and s.Consequently, the size of C is 3τ (σ + 1) + (2τ + 2)(σ + 1) Moreover, we require at most three layers to obtain ĥt j , 0 ≤ t ≤ τ − 1, for each j, 0 ≤ j ≤ σ, followed by the two layers containing r 0 , r 1 , . . ., r σ−1 and the top gate s.Thus, the depth is O(σ) = O( √ n).Finally, we show that the total wire length of C is O(n log n).To obtain the desired circuit layout, we use a number of duplicated copies of four basic layouts, A, B and C together with a layout D. An overview of the entire layout depicted with A, B, C, and D is shown in Fig 4 .We first evaluate the length of wires within each of the four layouts and then evaluate the length among the layouts.
In the layout A, we place the gates g t j , 0 ≤ t ≤ τ − 1, for every j, 0 ≤ j ≤ σ, as follows: we place the inputs x 0,j , x 1,j , . . ., x σ,j , vertically; moreover, to the right of the inputs, we place the gates g t j , 0 ≤ t ≤ τ − 1, each of which occupies σ vertical intersection points (see Fig 5(a)).The length of the wires comprising a gate is O(σ); therefore the length for all of the gates is O(στ ).The length of wires connected to each input is O(τ ); therefore the length for all of the gates is O(στ ).We consequently have wires of length O(στ ) in total for each layout.Similarly, the layout A is used to place the gates h t j , 0 ≤ t ≤ τ − 1, for every j, In the layout B, for every j, 0 ≤ j ≤ σ, we place the gates a j , b j , a t j and b t j , 0 ≤ t ≤ τ − 1, together with ĥt j , as follows.We place the gate b j at the bottom of the layout, and place a j just above b j ; each of them occupies 2τ horizontal intersection points.We then place a t j and b t j , 0 ≤ t ≤ τ − 1, each of which occupies two vertical intersection points.Finally, we place ĥt j , 0 ≤ t ≤ τ − 1, just above a t j , 0 ≤ t ≤ τ − 1, each of which occupies two vertical intersection points (see Fig. 6).The length of wires for composing a j (and b j ) is O(τ ), and the length for the other gates is O(1); therefore, the length for all the gates is O(τ ).The length of wires among a j , b j , a t j and b t j , 0 ≤ t ≤ τ −1, is clearly O(τ ), whereas the length of wires among b t j and ĥt j , 0 The length of wires for the outputs of ĥt j , 0 ≤ t ≤ τ − 1, is O(τ 2 ).Consequently, in total, we have wires of length O(τ 2 ) within each B. Since we have O(σ) copies of the layout B, we have wires of O(στ 2 ) length for the B's.
In the layout C, we place each of the gates r j , 0 ≤ j ≤ σ − 1, as follows.We place the gate r j so that each gate occupies τ horizontal intersection points and τ vertical intersection points (see Fig. 7).The length of wires for composing the gate is clearly O(τ ).Since we use O(σ) copies of the layout C, in total, wires of length O(στ ) for the C's.
In the layout D, we place the gate s as follows.The gate s occupies O(σ) horizontal intersection points (see Fig. 8).The length of wires for composing the gate s is clearly O(σ), and the length for the output of the gate is O(1).Consequently, in total, we have wires of length O(σ) within D.
Consider at last the wires on the outside of the layouts A, B, C and D. The length of a vertical wire between a pair of layouts A and C is O(στ ); since we have O(στ ) of such wires, the length of wires for all of the pairs among A's and C's is O(σ 2 τ 2 ).Similarly, we have wires of length O(σ 2 τ 2 ) for all of the

Lower Bound
In this section, we show that the size of the circuit given in Theorem 1 is optimal up to a polylogarithmic factor, as in the following theorem.
Theorem 2 Let C be an arbitrary threshold circuit computing P n D .Then C has size Ω( √ n/ log n).
It is known that any threshold circuit computing DISJ n has almost linear size in n. and Y = {y 0,0 , y 0,1 , . . ., y σ,σ } be sets of the input variables x = (x 0,0 , x 0,1 , . . ., x σ,σ ) and y = (y 0,0 , y 0,1 , . . ., y σ,σ ) to P n D .We define subsets X and Y of X and Y as (See Fig. 9.) We then fix every input in (X\X ) ∪ (Y \Y ) of C to 0. We denote the resulting circuit by D .Clearly, D computes P n D over X ∪ Y .By the definition of P n D , D * outputs 1 if and only if there exist indices i x and i y such that i x + 1 > i y and i x < i y + 1: i y − 1 < i x < i y + 1; therefore i x = i y .Consequently, D outputs 1 if and only if there exists an index i x , 0 ≤ i ≤ σ − 1, such that x i+1,i = y i,i+1 = 1; thus D computes the complement of DISJ σ , that is, DISJ √ n−1 .We complete the construction of D * by replacing the top gate g of D with a new gate g * computing its complement.Suppose g has threshold t and weight w 1 , w 2 , . . ., w z for a number z of inputs comprising x j+1,j and y j,j+1 , 0 ≤ j ≤ √ n − 2, together with the outputs of the gates in D * .We then replace g with g * with threshold −2t + 1 and weight −2w 1 , −2w 2 , . . ., −2w z .

Conclusion
In this paper, we consider a Boolean function P n D that models a simple information processing task on two-dimensional maps.We demonstrated that P n D is 130 Uchizawa et al.Threshold Circuits Detecting Global Patterns computable by a threshold circuit of size O( √ n log n), whereas any threshold circuit computing P n D requires size Ω( √ n/ log n).Our circuit has smaller size but greater depth than those given in [2].It would be interesting to discover a trade-off between the size and depth of threshold circuits computing P n D .

Figure 2 :
Figure 2: Input assignment (x, y) for P 49D , where (a) and (b) depict x and y, respectively.Note that we omit 0s in x and y for simplicity.In this case, P 49 D (x, y) = 1, since x 3,1 = 1 and y 1,2 = 1 (the corresponding locations are shaded).

Figure 4 :
Figure 4: Overview of the layout of C.