Tutorials Session A – Mechanisms Underlying Visual Object Recognition: Humans vs. Neurons vs. Machines

Visual object recognition (OR) is a central problem in systems neuroscience, human psychophysics, and computer vision. A recognition system must be robust to image variation produced by different “views” of each object– the so-called “invariance problem.” My laboratory aims to understand and emulate the primate brain’s solution to this problem.

We have previously shown that a part of the non-human primate ventral visual stream (inferior temporal cortex, IT) rapidly and automatically conveys neuronal population rate codes that qualitatively solve the invariance problem for vision. But are such codes quantitatively sufficient to explain behavioral OR performance? Our results show that these codes are a powerful object representation, in that low complexity decoding tools can be applied to them to perfectly predict human performance over a large range of OR tasks.

But how does the brain build this powerful representation? High-throughput computational methods can be used to explore a large family of biologically-constrained neural network architectures. Using this approach, we have recently discovered that functional optimization of this large family leads to specific algorithms that predict the response properties of IT dramatically better than all previous models. This suggests that these networks have captured key encoding mechanisms of human OR, and that today’s computer vision algorithms are very close to emulating the power of the primate OR system.

Speaker Details

Prof. DiCarlo received his Ph.D. in biomedical engineering and his M.D. from Johns Hopkins in 1998, and did his postdoctoral training in primate visual neurophysiology at Baylor College of Medicine. He joined the MIT faculty in 2002. He is a Sloan Fellow, a Pew Scholar, and a McKnight Scholar. His lab’s research goal is a computational understanding of the brain mechanisms that underlie object recognition. They use large-scale neurophysiology, brain imaging, optogenetic methods, and high-throughput computational simulations to understand how the primate ventral visual stream is able to untangle object identity from other latent image variables such as object position, scale, and pose. They have shown that populations of neurons at the highest cortical visual processing stage (IT) rapidly convey explicit representations of object identity, and that this ability is reshaped by natural visual experience. They have also shown how visual recognition tests can be used to discover new, high-performing bio-inspired algorithms. This understanding may inspire new machine vision systems, new neural prosthetics, and a foundation for understanding how high-level visual representation is altered in conditions such as agnosia, autism and dyslexia.

Date:
Speakers:
James J. DiCarlo
Affiliation:
Massachusetts Institute of Technology

    • Portrait of Jeff Running

      Jeff Running