ORIENTATION SELECTIVITY AND CORTICAL COMPUTATION

At the outset of this review, we suggested that orientation selectivity serves as a model system for understanding cortical computation. What conclusions can we draw from our view of the function of the striate cortex? Our survey suggests a set of provocative, if frankly speculative, ideas.

The three salient features of the model of cat layer 4 for which there is strong experimental evidence are orientation-specific feedforward excitation, strong push-pull inhibition, and a weaker recurrent excitation to amplify responses. The intracortical circuitry can be summarized as ``correlation-based'': excitatory cells connect to cells that are well correlated in activity; inhibitory cells connect to cells that are anti-correlated, or minimally coactive. Furthermore, a subset of the inhibitory cells must be more directly responsive to the inputs, and thus have broader tuning, than the excitatory cells. This suggests several candidate principles for the layer 4 cortical circuit. First, the entire circuit, including both feedforward and intracortical connections, can develop based on activity-dependent rules guided simply by the activity patterns of the feedforward inputs. Second, the circuit is very local: Cells need not integrate information even across an entire hypercolumn, but may restrict interactions to only a very local region, about 1/3 of a hypercolumn, representing $\pm 30^o$. Third, the pattern of activity is input-driven: e.g., inputs that stimulate cells with a broader or narrower range of preferred orientations elicit correspondingly broader or narrower activity patterns in the cortex. Fourth, the feedforward inhibition, which is directly driven by the input, is stronger than feedforward excitation and responds more like this input than do other cell. Studies in the rat whisker barrel system also indicate that the layer 4 computation is local, input-driven, and dependent on inhibitory responses that more directly reflect the thalamic input than do excitatory responses (Goldreich et al., 1999, Brumberg et al., 1996, Simons and Carvell, 1989, Pinto et al., 1996). It will be of great interest to determine if an analogue of push-pull inhibition can be found in layer 4 of this and other systems.

What computation might this circuit perform? It can allow layer 4 cells in visual cortex to recognize a given orientation independent of the stimulus contrast (Troyer et al., 1998). But this specific task can be abstracted to encompass more general rules of feature extraction, in particular the task of recognizing stimulus form independent of stimulu magnitude. Call the input set driving a given cell ${\cal A}$, which for a simple cell is the activity generated in the relay cells by optimally oriented light and dark bars in the ON and OFF subfields. Push-pull inhibition generalizes to inhibitory input from the pattern $\overline{\cal A}$ , the set of inputs most anticorrelated, or least coactive, with ${\cal A}$. For a simple cell, ${\cal A}$ is the same as $\overline{\cal A}$ except generated by stimuli with light and dark bars reversed. Finally, there is a large set of patterns, ${\cal B}$, that share some inputs with both ${\cal A}$ and $\overline{\cal A}$, but are uncorrelated -- only randomly coactive -- with each. In simple cells, these patterns correspond to input activity generated by stimuli of the orthogonal orientation. In simple cells receiving the input ${\cal A}$ alone (Figure 4D), we have seen that orientation selectivity becomes contrast dependent, because input pattern ${\cal B}$ of sufficiently large amplitude (an orthogonal stimulus of high contrast) can activate the cell. Adding strong push-pull inhibition translates into making the cell selective for the pattern $\left({\cal A} \mbox{\ AND NOT\ }\overline{\cal A}\right)$. As a result, ${\cal B}$ of any strength, since it activates both ${\cal A}$ and $\overline{\cal A}$ to some degree, can no longer activate the cell when push-pull inhibition is present. The cell becomes selective for pattern ${\cal A}$, independent of stimulus magnitude.

Thus, we postulate that layer 4 locally divides its inputs into opposing pairs of correlated input structures such that a cell responds only when one is present without the other. Layer 4, in turn, projects to layers 2/3 where in cat V1 we find complex cells that respond to a given stimulus orientation independent of its polarity. That is, while layer 4 cells seem to respond to $\left({\cal A} \mbox{\ AND NOT\ }\overline{\cal A}\right)$ layer 2/3 cells respond to something more like $\left({\cal A} \mbox{\ OR\ }\overline{\cal A}\right)$, extracting the element that the two opposites have in common (orientation), while discarding the elements that distinguish them (polarity). These ideas of opposition followed by synthesis as possible roots of mental processing are reminiscent both of many eastern philosophies and of ``dialectical'' western philosophies (e.g., Merleau-Ponty, 1962).

The feedback models, in contrast, incorporate a completely different philosophy of cortical processing. In these models, the cortex converges on stereotypical patterns of activity in response to a variety of stimuli. The architecture of the cortical circuit determines in advance how many different modes of response, and therefore how many different stimuli, can be encoded by the cortex. Those stimuli that do not conform to the predefined patterns will be represented as the nearest such pattern, and two or more patterns cannot easily be simultaneously represented. In the feed-forward model with strong push-pull inhibition, the cortex is more flexible in its response to the visual image. Different stimuli that evoke sufficiently different patterns of thalamic input will almost invariably evoke different patterns of cortical activity. Given the fundamental differences between the two models, determining which mode of operation the cortex uses (if indeed it uses either one) becomes all the more interesting.