Joint decoding of visual stimuli by IT neurons’ spike counts is not improved by simultaneous recording

Abstract

Information about visual stimuli such as objects and faces is represented across populations of neurons of the inferior temporal cortex. Does recording from inferotemporal neurons simultaneously tell you more than recording from them sequentially? Equivalently, are neurons conditionally independent given a stimulus? To evaluate these issues, we recorded from two monkeys during a passive viewing task. Multiple neurons were simultaneously recorded on separate electrodes. From spike counts in 50-ms windows, we computed the mutual information between counts and images for each neuron individually and jointly with other simultaneously recorded neurons. To determine the significance of these values, we shuffled the stimulus labels (to test if there was significant information) or shuffled responses across trials involving the same image (to see if there was synergistic coding). We recorded from 127 pairs of neurons where each neuron individually was visually responsive. Depending on the time window, we found up to ∼ 90% of these pairs showed significant information about the visual stimulus. Shuffling across trials failed to show evidence for synergistic coding. In summary, if you were given two of our neuronal responses and asked to guess the stimulus which produced them you could not, in principle, do better with two simultaneously recorded spike counts than with any two spike counts selected randomly from trials of the same type.

Appendix

Demonstration that

$$I(s;{\mathbf{r}}) - I(r_{1};s) - I(r_{2};s) = {\left\langle {I(r_{1};r_{2} |s)} \right\rangle}_{s} - I(r_{1};r_{2}).$$

Proof

$$\begin{aligned} I(s;{\mathbf{r}}) - I(r_{1};s) - I(r_{2};s) &{\mathop = \limits^a}{\sum\limits_{r_{1}, r_{2}, s}}P(s,r_{1}, r_{2})\log \frac{{P(s,r_{1}, r_{2})}}{{P(s)P(r_{1}, r_{2})}} - {\sum\limits_{r_{1}, s}}P(r_{1}, s)\log \frac{{P(r_{1}, s)}}{{P(r_{1})P(s)}}\\ &\quad - {\sum\limits_{r_{2}, s}}P(s,r_{2})\log \frac{{P(s,r_{2})}}{{P(s)P(r_{2})}}\\ &{\mathop = \limits^b}{\sum\limits_{r_{1}, r_{2}, s}}P(s,r_{1}, r_{2})\log \frac{{P(s,r_{1}, r_{2})}}{{P(s)P(r_{1}, r_{2})}} + {\sum\limits_{r_{1}, r_{2}, s}}P(r_{1}, r_{2}, s)\log \frac{{P(r_{1})P(s)}}{{P(r_{1}, s)}} \\ &\quad + {\sum\limits_{r_{2}, r_{1}, s}}P(s,r_{2}, r_{1})\log \frac{{P(s)P(r_{2})}}{{P(s,r_{2})}}\\ &{\mathop = \limits^c}{\sum\limits_{r_{1}, r_{2}, s}}P(s,r_{1}, r_{2})\;\log \frac{{P(s,r_{1}, r_{2})P(r_{1})P(s)P(s)P(r_{2})}}{{P(s)P(r_{1}, r_{2})P(r_{1}, s)P(s,r_{2})}}\\ &{\mathop = \limits^d}{\sum\limits_{r_{1}, r_{2}, s}}P(s,r_{1}, r_{2}){\left[ {\log \frac{{P(r_{1}, r_{2} |s)}}{{P(r_{1} |s)P(r_{2} |s)}} + \log \frac{{P(r_{1})P(r_{2})}}{{P(r_{1}, r_{2})}}} \right]}\\ &= {\sum\limits_s}P(s){\sum\limits_{r_{1}, r_{2}}}P(r_{1}, r_{2} |s)\log \frac{{P(r_{1}, r_{2} |s)}}{{P(r_{1} |s)P(r_{2} |s)}} - {\sum\limits_{r_{1}, r_{2}}}P(r_{1}, r_{2})\log \frac{{P(r_{1}, r_{2})}}{{P(r_{1})P(r_{2})}}\\ &{\mathop = \limits^e}\,{\left\langle {I(r_{1};r_{2} |s)} \right\rangle}_{s} - I(r_{1};r_{2})\\ \end{aligned}$$

where

a :

this follows from the definitions.

b :

this expands P(r ₁, s) as ${\sum\limits_{r_{2}} {P(r_{1}, r_{2}, s)}} \ldots,$ and pulls the minus signs inside the logarithms inverting their arguments.

c :

combines like terms.

d :

terms are recombined and we use the relationship that P(a| b)= P(a, b)/P(b).

e :

the result of applying the definitions for conditional mutual information and mutual information.