An organism actively determines which part of the world it samples
with its sensory organs.
A well known example is that animals orient their bodies, turn their
heads, or move their eyes in order to "look at" specific things in
their environment. How does this affect the structure of the signals
typically reaching the visual system? Given where you look, what does
your visual system see? Put another way, what is special about the
parts of real-world images that you choose to look at? Intuitively you
know the answer: you look at the the places which are
"interesting". But are "interesting" places different from "boring"
places in terms of quantifiable image properties? Can we make a
precise statement, in terms of information theory, about the
consequences of eye movements for the task of neurally encoding a
visual scene? I did some work on this question in collaboration with Tony Zador, then at the Sloan Center for Theoretical Neuroscience at the Salk Institute. We recorded the eye positions of human subjects as they look at pictures of real-world scenes. At right is an image from our ensemble with the eye positions of one subject shown in blue (the circles indicate the center of gaze every 20 msec). We have characterized how the regions in the images which subjects look at differ statistically from other image regions.
We found that the images falling in the fovea are higher in contrast than the rest of the image, less spatially correlated, and for both reasons, higher in information content (entropy). To explore the relative contributions of the physical properties of images versus the "cognitive content" of the images in determining eye movements, we used a variety of control images with more cognitive content (eg. faces) or less cognitive content (eg. fractals) as shown below.