Time course of visual perception: Coarse-to-fine processing and beyond

Our perception of a visual scene changes rapidly in time, even when the scene itself does not. It is increasingly clear that understanding how the visual percept changes in time is crucial to understanding how we see. We are still far from fully understanding the temporal changes in the visual percept and the neural mechanisms that underlie it. But recently, many disparate lines of evidence are beginning to converge to produce a complex but fuzzy picture of visual temporal dynamics. It is clear, largely from psychophysical studies in humans, that one can get the 'gist' of complex visual scenes within about 150 ms after the stimulus onset, even when the stimulus itself is presented as briefly as 10 ms or so. It generally takes longer processing, if not longer stimulus presentation, to identify individual objects. It may take even longer for a fuller semantic understanding, or awareness, of the scene to emerge and be encoded in short-term memory. Microelectrode recording studies in monkeys, along with neuroimaging studies mostly in humans, have elucidated many important temporal dynamic phenomena at the level of individual neurons and neuronal populations. Many of the temporal changes at the perceptual and the neural levels can be captured by the multifaceted and somewhat ambiguous concept of coarse-to-fine processing, although it is clear that not all temporal changes can be characterized this way. A more comprehensive, albeit unproven, alternative framework for understanding visual temporal dynamics is to view it as a sequential, Bayesian decision-making process. At each step, the visual system infers the likely nature visual scene by jointly evaluating the available processed image information and prior knowledge about the scene, including prior inferences. Whether the processing proceeds in a coarse-to-fine fashion depends largely on whether the underlying computations are hierarchical or not. Characterizing these inferential steps from the computational, perceptual and neural standpoints will be a key part of future work in this emerging field.